Update: Microsoft will be moving away from UserVoice sites on a product-by-product basis throughout the 2021 calendar year. We will leverage 1st party solutions for customer feedback. Learn more here.

Azure Databricks

Azure Databricks is an Apache Spark-based analytics platform optimized for the Microsoft Azure cloud services platform. Designed with the founders of Apache Spark, Databricks is integrated with Azure to provide one-click setup, streamlined workflows, and an interactive workspace that enables collaboration between data scientists, data engineers, and business analysts.

We would love to hear any feedback you have for Azure Databricks.
For more details about Azure Databricks, try our documentation page.

  1. Access Azure Certificate Stored in Azure KeyVault inside Databricks

    What is option to access Certificate stored in Azure KeyVault. Need some feature like dbutil.secrete.

    4 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  2. Add transport of all Cluster events to Log Analytics workspace

    Currently diagnostic setting for a workspace allows only limited number of events to be transported into Log Analytics DatabricksClusters category. Events that are missing ie:


    • cluster termination

    • cluster auto-sizing related event (adding machines, expanding disks and opposite)

    In general it would be more than welcome to have all of the information available in cluster Event log to be made available in Log Analytics as well

    4 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  3. Enable ADLS passthrough and table access control together.

    Currently, in high concurrency cluster with ADLS passthrough enabled, table access control can't be enabled at the same time. If database/tables are created on top of storage containers and files in storage containers have ACL/RBAC applied, it works fine and users can't access underlying data if RBAC/ACL on the data files don't allow the access. If a user tries to run a select query on a table with underlying data in a container the user doesn’t have access to, the select query returns an error as expected.

    Users can still view the tables and drop tables and databases not created…

    3 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  4. Allow owner of a database to manage tables (alter/delete etc) that are not owned by them.

    I would like our Databricks environment be as self service as possible. Now that some users are transitioning, it is clear that only an administrator can drop a table or change owner for a user that is no longer here. I would like to be able to grant this privilege to the owner of the database. If I did not want the owner of a database to be able to delete/alter tables, I would have assigned one of our administrators.

    3 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  5. When are you adding an API to allow for Databricks Secret Scopes(back by Keyvault) to be created?

    Currently this can only be done by the UI, it can not be automated as the CLI doesnt support it either.

    3 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  6. Access to ADSL Gen 2 using Certificate based Service Principal

    How to access ADSL Gen 2 using Certificate based Service principal. CUrrently ADSL Gen 2 access is supported via Secret\Key based service principal. But our All Service Principals are based on Certificates which are stored in KeyVault.

    3 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  7. Provide Option to connect Azure Synapse using service principal

    Provide Option to connect to Azure Synapse using Azuer Service principal. The current connector com.databricks.spark.sqldw, doesnt have this option.

    3 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  8. Equip a lock button on notebook for GitHub(Devops) management

    When users use Azure Data Factory to invoke Databricks notebook activity, it will use current code instead of committed version of notebook. it would be recommended to add the feature to lock the current notebook, and let Azure Data Factory use committed one in Github or Devops.

    3 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  9. Azure Databricks logging, add notebook id to every command in the stderr

    Add in the notebook id and timestamp on the stderr log. Command ID is already inserted into the logs.

    2 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  10. Don't erase prints when returning an output message with dbutils.notebook.exit(msg)

    If we have a job that prints something, but at the end we use the command dbutils.notebook.exit("Hello World") (in Python) to return a value to whoever runs the job, the prints are erased. Please, keep the prints even if we use exit. This seems like a bug to me, I want to be able to read the logs anyway.

    2 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  11. Provide a warning that editing a notebook associated with a Job in the UI will also result in associated parameters being lost

    The UI has an Edit option on the Notebook associated with a job, when this is edited any associated parameters are lost, without any warning being given.
    There are scenarios where the Notebook path has to be edited but the parameters are required to remain the same (for example if workspaces have been re-organised).
    If the user isn’t aware that editing the path to the notebook also removes the parameters, this can result in having to re-type what can sometimes be extensive parameter details.
    In these cases a warning would be very helpful.

    2 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  12. Enable subnets smaller than /26

    As far as I can find there was an arbitrary decision made that /26 is the smallest subnet for databricks. Is there a reason for this? Otherwise this is going to eat up addresses quickly when deploying many environments of databricks.

    Can smaller subnets like /28 be enabled for this product?

    2 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  13. Multiselect widget with list type default value

    The Multiselect widget currently doesn't support selecting more than one default value. There is also no "select all" option so selecting all or many options is dumbersome. Also, this means that the return value can be either a string or a list of strings. If the default value was a list type, these issues would be easily solved.

    2 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  14. Button to just restart the interpreter

    For example for Python Notebooks, it would effectively run dbutils.library.restartPython()

    2 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  15. Support Unit testing with databricks-connect

    We've run into an issue where we're using databricks-connect in a python project for remote connection to a Databricks cluster.

    In the project we have unit tests for our project configured to run in local mode similar to the approach described here: https://databricks.com/session/mastering-spark-unit-testing

    The issue we're running into is that databricks-connect has no support for running with a local master. Also, since it is an API equivalent replacement for pyspark we are unable to include pyspark in the dependencies. We're using Pipenv for environment management, but I think this would be a similar problem with other environment management tools.

    This…

    2 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  16. Add cluster with RStudio installed the ability to record and store inactivity logs and also work with the auto-terminate option enabled

    Add cluster with RStudio installed the ability to record and store inactivity logs and also work with the auto-terminate option enabled.

    2 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  17. Multiple Cron schedules for one job

    Many simple schedules cannot be configured for a single job in Databricks due to the limitations of cron schedules. E.g. running a job every 40 minutes. Multiple schedules could provide such a frequency, but today, that would require having duplicate copies of the same job with different schedules.

    2 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  18. runtime versions

    Databricks needs to better test the compatibility between different runtime versions and the various packages.

    2 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  19. Timestamp for each command

    It would be very helpful to see the exact timestamps for when a command started and finished processing, not only the runtime in msec.

    2 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  20. Support for enqueuing job clusters onto an instance pool

    Using instance pools to optimize the runtime of smaller, automated jobs is currently at odds with the built-in scheduling system.

    This is because an instance pool will simply reject a job for which it can't immediately procure the required amount of nodes.

    This is a proposal to have an enqueuing behavior such that an automated job will instead wait (possibly with a configurable upper time limit) for resources to become available. The requirement would then be that either the minimum autoscaling configuration or the fixed number of nodes would be satisified at the time of job start.

    Having this functionality…

    2 votes

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Azure Databricks

Categories

Feedback and Knowledge Base