Azure Databricks

Azure Databricks is an Apache Spark-based analytics platform optimized for the Microsoft Azure cloud services platform. Designed with the founders of Apache Spark, Databricks is integrated with Azure to provide one-click setup, streamlined workflows, and an interactive workspace that enables collaboration between data scientists, data engineers, and business analysts.

We would love to hear any feedback you have for Azure Databricks.
For more details about Azure Databricks, try our documentation page.

  1. H2O Flow or H2O Flow Equivalent in Azure

    The H2O Flow GUI works with Databricks AWS but not Azure. It auto populates a huge amount of interactive charts and metrics for post model evaluation via a local host URL. I would like to request H2O Flow be enabled in Azure, but further request that Databricks add many more interactive auto generated content like H2O Flow does. This includes interactive ROC curves where you can traverse the confusion matrix by selecting any point on the curve, cross validation data set score, and variable importance.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  2. Allow tag modification in the managed resource group

    We have noticed that every managed resource group of every Databricks workspace has a deny assignment that prevents from modifying the RG's tags, after the creation. This often cause problems with governance and monitoring of the managed resources. Its purpose doesn't seem to be documented anywhere, and it doesn't seem like it can be bypassed.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. allow sandboxes to allow new users to follow this through

    allow sandboxes to allow new users to follow this through

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. Full Tech Support Required for Customers using .Net for Spark

    I opened a support ticket about .Net for Spark to ask what level of support was offered, and if this was ready for use in production. I was told that we can use .Net for Spark in Azure Databricks but it is not formally supported by Microsoft.

    Instead of contacting Azure support, they suggested that I work with the open source community at the github repo: https://github.com/dotnet/spark

    Ideally Microsoft would give this effort its formal support, as would Databricks. A lot of .Net shops would feel more comfortable about using .Net in a Databricks cluster if there was a well-defined…

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  5. Jobs/Interactive Tasks Prioritization

    By default, when a new task comes in, it uses all of the available resources in Databricks and other tasks might need to wait. As this is Big Data and shared environment, most of the times, I think it will be beneficial to be able to assign priorities at Job Level or Task level (when working Interactively in a Notebook). This way, you will instruct Databricks that if a task with higher priority comes in, it has to yield some the currently allocated resources and grant the required memory/cpu to the task with higher priority. If many users are working…

    8 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. Improvements needed in Databricks diagnostics settings

    Improvements needed in Databricks diagnostics settings,

    Log analytics table name -DatabricksJobs

    1.Include the Job name in the logs
    2.Include job run time duration in logs

    Log analytics table name -DatabricksClusters

    1.With respect to Databricks clusters, the logs are populated only if there’s an operation performed.
    2.To know the availability status of the cluster, it would helpful to have some kind of an availability log entry once every 5 minutes or once a minute.
    3.Along with the availability log, it would also be helpful to know the number of nodes that are currently running in the cluster. The existing log entry…

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  7. Create PaaS-managed Databricks Service Tags for use in NSGs instead of VirtualNetwork

    The default, highest priority NSG rule created by Databricks in PaaS-managed NSGs is "permit any protocol from VirtualNetwork to VirtualNetwork". The rule is enforced with a network intent policy and cannot be overridden. In cases where UDRs with default routes (destination 0.0.0.0/0) are attached to the Databricks subnets, or the VNets learn a default route from a VNet gateway, the NSG effectively becomes a "permit any any" rule. Databricks nodes have public IP addresses, which creates an unreasonable surface attack area when combined with a wide open NSG. Having a PaaS-managed service tag to permit required internal network access without…

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  8. Databricks-Connect with multi configuration on same machine

    Is the databricks-connect library (https://docs.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect) have option to support multiple configuration? Our scenario require switch from one configuration to another one when necessary. The current doc seems indicate it only support one global settings and we tested to Anaconda's environment and settings configured under one environment been used by another one.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  9. Website typo

    See the attached image for a typo on the Databricks website, on the page for cluster details.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  10. a button to clear all failed commands listing on the side

    a button to clear all failed commands listing on the side

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. Support library/wheel override

    I have one databrick cluster setup to run jobs, each time I submit a job, I specify the wheel file in the libraries arguments pass to Databrick. It's only work the first time. The following jobs will receive message like following:
    20/06/16 02:47:05 WARN SparkContext: The jar /local_disk0/tmp/AI..-2.0.0-py3-none-any.whl has been added already. Overwriting of added jars is not supported in the current version.

    Since I often need to modify above AI..-2.0.0-py3-none-any.whl, it will be great to allow for overwriting library with same name for each job execution

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  12. Generalize AuthN passthrough for specific AuthZ endpoints

    Currently the credential passthrough feature/s offer limited utility.

    Aim: generalize the feature so that an AD authenticated user (presumably with bearer token) can access a wide variety of Azure service end-points with great admin configurability.

    Most end points should be part of the regular service with some needing premium subscription.

    There are many such endpoints. The current one we're encountering is the Azure Artifacts Feed end-point. With the ability to configure automatic AuthZ (bearer token -> access token) we get per-notebook installations of packages with per-user credentialing. This appears to align well with V7ML's addition of %pip and %conda magic…

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  13. Microsoft.DataBricks/workspaces/parameters.storageAccountName does not work.

    Microsoft.DataBricks/workspaces/parameters.storageAccountName is not working. Can you please bring this feature back? We like to keep our naming conventions standards.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. canceling a command could show the output of the last run

    It could be nice if canceling a command before it starts to execute - will show the output of the last run - as sometimes i forget to save it somewhere and then i accidentally run the command again and the output is cleared.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  15. Query all table ACLs for a principal, database etc.

    it would be really helpful to be able to retrieve all the explicit grants that a principal has

    at present you can:
    show grant <principal> on <data_object>

    in order to be able to view all the permissions a user has it would be good to:
    show grant <principal>
    or
    show grant <principal> on *
    or
    show all grants for <principal>

    and a case where you would like to get ACLs of DB and all objects beneath like:

    show grant on DATABASE recursive

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. Add cluster with RStudio installed the ability to record and store inactivity logs and also work with the auto-terminate option enabled

    Add cluster with RStudio installed the ability to record and store inactivity logs and also work with the auto-terminate option enabled.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  17. Official PowerShell module for Azure Databricks

    An official PowerShell module for Databricks management would be very useful. There are some community created modules which are not being updated on time. Since PowerShell can run on both Windows and Linux this could be universal tool for interacting with API which could reduce the development time for all CI/CD tasks.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  18. Log automated deletion of unpinned clusters

    If unpinned clusters are deleted by databricks, there are no logs to be found anywhere in Log Analytics. For auditing and monitoring purposes i think this would be needed.

    Sometimes people forget to pin clusters and then you at least want to know from the logs what happened to it and when it got deleted.

    6 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  19. Access for internal/private maven repositories from Databricks

    Need a support to pull the libraries from internal/private maven repositories from databricks.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  20. 1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
← Previous 1 3 4 5 6 7 8
  • Don't see your idea?

Azure Databricks

Categories

Feedback and Knowledge Base