Databricks Service principal per workspace for specific KeyVault access
Databricks currently accesses KeyVault from the control plane and uses the same AzureDatabricks Service principal for ALL databricks workspaces in the tennant.
At present, if you create a secret scope in workspace A on KeyVault A and a new secret scope in workspace B on KeyVault B then the Azure databricks service principal will have access to both keyvaults. Therefore, providing you are privielaged enough to know the details (resource uri) of the keyvaults then you can create a scope from your own databricks workspace C and get access to all the keys!!
It should be possible to specify an individual service principal for a specific databricks workspace and give that specific permissions on the KeyVault.
Abhinav Garg commented
Thanks for posting this. The AzureDatabricks service principal is basically a replicated first-party service SP. And as you indicated, it's used under the hood to access the AKV instance.
We do have a requirement that anyone creating the secret scope in a workspace have at least contributor-level permissions on the AKV instance. So if somebody is being granted that permission, that is actually a cloud RBAC governance construct.
Leaving Azure Databricks aside for a while, the specific problem you've indicated would apply to any service/application that would use that AKV instance. Even if we allow you to specify a separate SP (or managed identity) per secret scope or workspace, a privileged user with the contributor level permission on the AKV instance could still access it directly. I agree with that requirement in general so as to have proper segmentation between the workspaces, but it doesn't take away the need to be buttoned up from a cloud IAM / RBAC perspective.
We've implemented Secret Access Control - https://docs.microsoft.com/en-us/azure/databricks/security/access-control/secret-acl for controlling access to a secret scope. Ideally only platform / workspace admins should've permissions to configure AKVs and secret scope, and then they could apply secret ACLs within the workspaces.
Please feel free to comment further in case there're any holes in the above explanation. This is a good ask. I just want to qualify it more so we're not solving for meta-cloud provider problems in the product. Thanks.
This seems like a huge security concern