Azure Databricks

Azure Databricks is an Apache Spark-based analytics platform optimized for the Microsoft Azure cloud services platform. Designed with the founders of Apache Spark, Databricks is integrated with Azure to provide one-click setup, streamlined workflows, and an interactive workspace that enables collaboration between data scientists, data engineers, and business analysts.

We would love to hear any feedback you have for Azure Databricks.
For more details about Azure Databricks, try our documentation page.

  1. Access for internal/private maven repositories from Databricks

    Need a support to pull the libraries from internal/private maven repositories from databricks.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  2. Modify the /secrets/scopes/create API so we can add Key Vault backed scopes

    Currently, we can only add a Databricks-backed secret Scope to the workspace using the REST api.

    I want to deploy workspaces programatically, so I would like to be able to deploy a Key Vault backed secret Scope through the API.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  3. Access to ADSL Gen 2 using Certificate based Service Principal

    How to access ADSL Gen 2 using Certificate based Service principal. CUrrently ADSL Gen 2 access is supported via Secret\Key based service principal. But our All Service Principals are based on Certificates which are stored in KeyVault.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  4. Provide Option to connect Azure Synapse using service principal

    Provide Option to connect to Azure Synapse using Azuer Service principal. The current connector com.databricks.spark.sqldw, doesnt have this option.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  5. Workspace, cluster, job ownership changes

    It is unfortunate situation where there is no possibility to change ownership of:
    * workspace
    * cluster
    * job
    People are leaving companies, all of the above reside as owned by them. I understand that some of the issues regarding individuals owning resources can be avoided with SP, but it can be a solution for new deployments and still can be inconvenient in some use cases. There should be an option to change ownership. Lack of this feature also may produce an issue for operation of clusters and jobs:
    * cluster whose owner is removed from workspace will not be…

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. Cluster status API - check if cluster is up and running

    Currently there is no way to use API to verify if a given cluster is running or not. It causes a lot of trouble for tools like PowerBI to verify upfront if queries can run or not. There is such information available on the GUI, so we would gladly welcome same functionality to be with the REST API.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  7. runtime versions

    Databricks needs to better test the compatibility between different runtime versions and the various packages.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  8. Timestamp for each command

    It would be very helpful to see the exact timestamps for when a command started and finished processing, not only the runtime in msec.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  9. ARM templet with databricks within vnet can only be deployed once

    Cannot redeploy arm template with databricks with vnet integration. More details here: https://github.com/Azure/azure-quickstart-templates/issues/6670

    Actually I get the same error when issueing az network nsg create command for already existing nsg (used by databricks).

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  10. NCv3 in North Europe

    Tesla K80 cards are super slow and lack mixed precision support, furthermore, they're incompatible with Nvidia RAPIDS, because they're very old.
    Google doesn't even give away K80 for free in colab unless all P100's are taken.
    Having only K80s available in north europe is outrageos.
    We've had to shuffle data and models back and forth between north and west to train models in west. This is becoming very cumbersome and wasting a lot of time for us.
    Please add contemporary GPUs in North Europe

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. Support for enqueuing job clusters onto an instance pool

    Using instance pools to optimize the runtime of smaller, automated jobs is currently at odds with the built-in scheduling system.

    This is because an instance pool will simply reject a job for which it can't immediately procure the required amount of nodes.

    This is a proposal to have an enqueuing behavior such that an automated job will instead wait (possibly with a configurable upper time limit) for resources to become available. The requirement would then be that either the minimum autoscaling configuration or the fixed number of nodes would be satisified at the time of job start.

    Having this functionality…

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  12. Active Directory authentication enabled on SQL connector (azure datawarehouse, sql server, etc. )

    The spark SQL connector in scala and python should be able to connect through active directory authentication.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  13. Support for ELK stack and Kubernetes on Databricks cluster

    Can we support ELK stack and Azure kubernetes on the databricks cluster so that we can solve the application portal and search use case on datastore in databricks.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. Be able to jupyter/jupyter lab frontend also

    Although databricks notebooks are good, at some tasks even better than jupyter notebooks, but still miss a lot of common tasks, shortcuts, extensions. Would be great to be able to opt for jupyter or jupyter hub
    also see:
    http://feedback.databricks.com/forums/263785-product-feedback/suggestions/10703517-databricks-could-use-jupyter-as-a-front-end

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  15. Run Jar like just other programs but not library for reuse

    I have algorithms written in jar and compiled as JAR.
    I would like to submit them on ML cluster not as a library but just another program(notebook scripts). Upgraded/modified Jar would need not then require clean up of already installed older version from the cluster.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. Connection to Azure Data Lake Analytics Tables/views

    We have business data objects (tables / views) created on ADLA and we want to call them from Databricks can we have a connection same as ADLS.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  17. Data Bricks should be able to be integrated with Service Now to analyze past product problems

    DataBricks can be integrated to streaming data and product master data to get the master data as well as performance data of the device. But it should be able to integrate with Service now to query and auto suggest problems analyzing the telemetry of the device to suggest that there is a problem and suggest solution from past incidents.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  18. Provide support for mrsdeploy on Azure databricks

    I wish to deploy R model to Azure ML server using mrsdeploy from Azure databricks

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  19. H2O Flow or H2O Flow Equivalent in Azure

    The H2O Flow GUI works with Databricks AWS but not Azure. It auto populates a huge amount of interactive charts and metrics for post model evaluation via a local host URL. I would like to request H2O Flow be enabled in Azure, but further request that Databricks add many more interactive auto generated content like H2O Flow does. This includes interactive ROC curves where you can traverse the confusion matrix by selecting any point on the curve, cross validation data set score, and variable importance.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  20. Allow tag modification in the managed resource group

    We have noticed that every managed resource group of every Databricks workspace has a deny assignment that prevents from modifying the RG's tags, after the creation. This often cause problems with governance and monitoring of the managed resources. Its purpose doesn't seem to be documented anywhere, and it doesn't seem like it can be bypassed.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Azure Databricks

Categories

Feedback and Knowledge Base