Enable GPU support when deploying to Azure Container Instances
Azure Container Instances can be configured to provide GPU resources to the container (https://docs.microsoft.com/en-us/azure/container-instances/container-instances-gpu). However, there is no way to ask for GPU resources when deploying ML services via the Azure ML SDK (AciWebservice.deploy_configuration) or Azure CLI (az ml service create aci). Even if the image is built with GPU support enabled (ContainerImage.image_configuration(enable_gpu=True,...)), the image does not work properly in ACI because the GPU resources are not present.
Please extend the SDK and CLI to allow specifying the GPU count and SKU when deploying to ACI.
Jakub Bereżański
shared this idea
1 comment
-
Jakub Bereżański
commented
The image does work correctly in ACI with GPU support when deployed "manually" via "az container create", but then it is obviously not registered as a webservice in the ML Workspace and it is missing all features provided by the front-end sidecar container deployed by the ML Service (TLS, authentication, easy access to logs).