Azure Databricks

Azure Databricks is an Apache Spark-based analytics platform optimized for the Microsoft Azure cloud services platform. Designed with the founders of Apache Spark, Databricks is integrated with Azure to provide one-click setup, streamlined workflows, and an interactive workspace that enables collaboration between data scientists, data engineers, and business analysts.

We would love to hear any feedback you have for Azure Databricks.
For more details about Azure Databricks, try our documentation page.

  1. Support Single-Sign On with custom identity providers

    Databricks on AWS already supports multiple identity providers for SSO. Check https://docs.databricks.com/administration-guide/users-groups/single-sign-on/index.html.

    There is no reason why Azure Databricks should be limited only to AAD for SSO.

    5 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  2. Personal Access Token Security Improvements

    Most Databricks users end up needing to generate a Personal Access Token - which I am guessing is why Microsoft started to default that setting to ON.

    The problem is, from an Access Control perspective these tokens present a massive risk to any organization because there are no controls around them.

    These tokens allow direct access to everything the user has access to and all it takes to cause a major data breach is for one user to accidentally post one of these tokens on a public forum or GitHub.

    Here are a few specific issues:
    1. Even though conditional…

    5 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  3. Please add feature that can use Table access control when use "R"

    At now, we can use Table access control with python and SQL only .
    So, please add feature that can use Table access control when use "R".

    5 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. Sending out email inside of Databricks Notebook

    We need a way to send out email from databricks notebook.

    5 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    3 comments  ·  Flag idea as inappropriate…  ·  Admin →
  5. Databricks ControlPlaneIp & WebappIp as ServiceTags in NSG/Azure Firewall

    It would be nice if there was a ServiceTag for the Databricks Control Plane and Webapp IP ranges.
    Thanks.

    5 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. diff over multiple versions in the "Revision history" and/or export diffs

    The "Revision-History" is very unhandy when seraching for changes.
    a) The view renders very slowly - even worse for bigger notebooks
    b) The diffs have to be searched by scrolling though the whole notebook looking for red an green areas
    c) Worst of all: I can only compare one revision to the very one before
    There is an urgent need of faster comparison - even a simple export of the revisions in text-format/unix-diff would increase the value of this basically nice front-end feature a lot. We are all used to classic diff-tool-options like scrollbar-marking, jumping to next change and mainly…

    4 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  7. Integration with IntelliJ

    As a developer/Data Scientists, I would like to
    1. develop Apache Spark applications written in Scala, and then submit them to an cluster directly from the IntelliJ.
    2. Access Databricks cluster resources.
    3. Develop and run a Scala Spark application locally
    If the algorithm is upgraded, I need to rerun the algorithm with changes from Intelij locally without the need of removing older versions like libraries(jar) from cluster.

    4 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  8. SSH databricks

    I hope to access databricks via SSH like HDinsight.

    4 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  9. Access Azure Certificate Stored in Azure KeyVault inside Databricks

    What is option to access Certificate stored in Azure KeyVault. Need some feature like dbutil.secrete.

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  10. Equip a lock button on notebook for GitHub(Devops) management

    When users use Azure Data Factory to invoke Databricks notebook activity, it will use current code instead of committed version of notebook. it would be recommended to add the feature to lock the current notebook, and let Azure Data Factory use committed one in Github or Devops.

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  11. NAT Gateway Compatibility

    Make Databricks Workspaces Compatible with NAT Gateways. Currently when you associate a NAT Gateway with the public subnet of the Databrick workspace clusters will not start and raise the following error:

    Azure error code: AzureVnetConfigurationFailure(SubnetWithNatGatewayAndBasicSkuResourceNotAllowed)
    Azure error message: Encountered error while attempting to create NIC within injected virtual network. Details:
    NAT Gateway cannot be deployed on subnet containing Basic SKU Public IP addresses or Basic SKU Load Balancer.

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  12. Add transport of all Cluster events to Log Analytics workspace

    Currently diagnostic setting for a workspace allows only limited number of events to be transported into Log Analytics DatabricksClusters category. Events that are missing ie:


    • cluster termination

    • cluster auto-sizing related event (adding machines, expanding disks and opposite)

    In general it would be more than welcome to have all of the information available in cluster Event log to be made available in Log Analytics as well

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  13. Moving multiple cells up/down together

    In jupyter notebook, you are able to select multiple cells and easily move them up/down together. This functionality is not possible right now in databricks.

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. Allow to categorize Azure Databricks costs by cluster name

    Allow to see how much each cluster are spending, so we can manage better the costs relative to certain activities

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  15. Accessing ADLS Gen 2 backed tables via ODBC

    It seems that Azure Databricks does not support accessing tables backed by ADLS Gen 2 via ODBC or Power BI. It works fine if we use blob storage. It gives an error - "java.util.concurrent.ExecutionException: java.io.IOException: There is no primary group for UGI (Basic token).

    ADLS Gen 2 tables can be accessed when using Notebook.

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  16. Connector for Azure cosmos db (mongo DB API) from azure databricks spark

    Please provide some details to connector for spart structured streaming to azure cosmos db (MongoDB API) by using Azure databricks spark

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  17. Programmatically Access Cluster/Jobs Access Controls

    Currently, to change/add access controls on a job or cluster you need to go into the portal and do it manually. It should be possible to provide access controls in the cluster/jobs create payload so that they are automatically there when the cluster/job spins up.

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  18. Support conditional access

    I really want to use conditional access in conjunction with Azure AD authentication to restrict the login locations. This does not seem possible today.

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  19. Any update on HIPAA + HITRUST compliance with this offering?

    Our team works in healthcare and this could be something to look into if it meets all the compliance checks.

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    3 comments  ·  Flag idea as inappropriate…  ·  Admin →
  20. Create PaaS-managed Databricks Service Tags for use in NSGs instead of VirtualNetwork

    The default, highest priority NSG rule created by Databricks in PaaS-managed NSGs is "permit any protocol from VirtualNetwork to VirtualNetwork". The rule is enforced with a network intent policy and cannot be overridden. In cases where UDRs with default routes (destination 0.0.0.0/0) are attached to the Databricks subnets, or the VNets learn a default route from a VNet gateway, the NSG effectively becomes a "permit any any" rule. Databricks nodes have public IP addresses, which creates an unreasonable surface attack area when combined with a wide open NSG. Having a PaaS-managed service tag to permit required internal network access without…

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Azure Databricks

Categories

Feedback and Knowledge Base