Databricks Service principal per workspace for specific KeyVault access
Databricks currently accesses KeyVault from the control plane and uses the same AzureDatabricks Service principal for ALL databricks workspaces in the tennant.
At present, if you create a secret scope in workspace A on KeyVault A and a new secret scope in workspace B on KeyVault B then the Azure databricks service principal will have access to both keyvaults. Therefore, providing you are privielaged enough to know the details (resource uri) of the keyvaults then you can create a scope from your own databricks workspace C and get access to all the keys!!
It should be possible to specify an individual service principal for a specific databricks workspace and give that specific permissions on the KeyVault.
@Abhinav Garg - you are right but only partially.
Lets say one has had a contributor role on a key vault at the scope creation time. Later on, that role was removed for them. Now, even though the person who created the secret scope does not have access to the key vault anymore, the clusters in data bricks workspace still do. And it's impossible to remove access just for a given workspace - you can only remove access for all of them! And this is usually an issue since people often have separate data bricks workspace per dev/qa/uat/prod environments.
Imo there should be a separate SPN per data bricks workspace. And I agree that the current design raises some serious security concerns.
Abhinav Garg commented
Thanks for the feedback. Just to clarify, this is incorrect - "providing you are priveleged enough to know the details (resource uri) of the keyvaults then you can create a scope from your own databricks workspace C and get access to all the keys".
One needs to be a Contributor on the Azure Key Vault instance to be able to create the secret scope from any workspace. So it really comes down to how good or weak the RBAC permission model is within an organization, which applies to "any" resource in the subscription. Kindly refer to this - https://docs.microsoft.com/en-us/azure/key-vault/general/overview-security#managing-administrative-access-to-key-vault.
David Beavon commented
This is really crazy. Please fix.
We have several instances of databricks per environment (dev,test,prod) and do NOT want the developers on a development instance of databricks to have access to production keys just because they are working in the same tenant.
This seems like a huge security concern