Enable GPU support when deploying to Azure Container Instances
Azure Container Instances can be configured to provide GPU resources to the container (https://docs.microsoft.com/en-us/azure/container-instances/container-instances-gpu). However, there is no way to ask for GPU resources when deploying ML services via the Azure ML SDK (AciWebservice.deploy_configuration) or Azure CLI (az ml service create aci). Even if the image is built with GPU support enabled (ContainerImage.image_configuration(enable_gpu=True,...)), the image does not work properly in ACI because the GPU resources are not present.
Please extend the SDK and CLI to allow specifying the GPU count and SKU when deploying to ACI.
Jakub Bereżański
shared this idea
Hi Azure Customer,
Thank you for your feedback. This feature is currently not planned mainly because ACI doesn’t have full GPU support yet. We will follow up with ACI team and expose it when they add full support. At the meantime, we will keep this feedback and vote opening.
Thank you for your understanding.
Regards,
Azure CXP Community
1 comment
-
Jakub Bereżański
commented
The image does work correctly in ACI with GPU support when deployed "manually" via "az container create", but then it is obviously not registered as a webservice in the ML Workspace and it is missing all features provided by the front-end sidecar container deployed by the ML Service (TLS, authentication, easy access to logs).