Azure Databricks

Azure Databricks is an Apache Spark-based analytics platform optimized for the Microsoft Azure cloud services platform. Designed with the founders of Apache Spark, Databricks is integrated with Azure to provide one-click setup, streamlined workflows, and an interactive workspace that enables collaboration between data scientists, data engineers, and business analysts.

We would love to hear any feedback you have for Azure Databricks.
For more details about Azure Databricks, try our documentation page.

  1. Allow to categorize Azure Databricks costs by cluster name

    Allow to see how much each cluster are spending, so we can manage better the costs relative to certain activities

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  2. Accessing ADLS Gen 2 backed tables via ODBC

    It seems that Azure Databricks does not support accessing tables backed by ADLS Gen 2 via ODBC or Power BI. It works fine if we use blob storage. It gives an error - "java.util.concurrent.ExecutionException: java.io.IOException: There is no primary group for UGI (Basic token).

    ADLS Gen 2 tables can be accessed when using Notebook.

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  3. Connector for Azure cosmos db (mongo DB API) from azure databricks spark

    Please provide some details to connector for spart structured streaming to azure cosmos db (MongoDB API) by using Azure databricks spark

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. Integration with IntelliJ

    As a developer/Data Scientists, I would like to
    1. develop Apache Spark applications written in Scala, and then submit them to an cluster directly from the IntelliJ.
    2. Access Databricks cluster resources.
    3. Develop and run a Scala Spark application locally
    If the algorithm is upgraded, I need to rerun the algorithm with changes from Intelij locally without the need of removing older versions like libraries(jar) from cluster.

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  5. Programmatically Access Cluster/Jobs Access Controls

    Currently, to change/add access controls on a job or cluster you need to go into the portal and do it manually. It should be possible to provide access controls in the cluster/jobs create payload so that they are automatically there when the cluster/job spins up.

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. Any update on HIPAA + HITRUST compliance with this offering?

    Our team works in healthcare and this could be something to look into if it meets all the compliance checks.

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    3 comments  ·  Flag idea as inappropriate…  ·  Admin →
  7. Implement access token auto refresh when using credential passthrough

    When a cluster is configured with credential passthrough we are getting an access denied error after 1 hour of running a notebook due to the AD access token expiration. Because of that, it would be nice to have the access token auto refresh feature, with no need to an Azure Active Directory admin increase the AccessTokenLifetime for users.

    This feature is also cited in a comment here: https://feedback.azure.com/forums/909463-azure-databricks/suggestions/36879865-enable-azure-ad-credential-passthrough-to-adls-gen

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  8. Timestamp for each command

    It would be very helpful to see the exact timestamps for when a command started and finished processing, not only the runtime in msec.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  9. Active Directory authentication enabled on SQL connector (azure datawarehouse, sql server, etc. )

    The spark SQL connector in scala and python should be able to connect through active directory authentication.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  10. Sending out email inside of Databricks Notebook

    We need a way to send out email from databricks notebook.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. Support for ELK stack and Kubernetes on Databricks cluster

    Can we support ELK stack and Azure kubernetes on the databricks cluster so that we can solve the application portal and search use case on datastore in databricks.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  12. Be able to jupyter/jupyter lab frontend also

    Although databricks notebooks are good, at some tasks even better than jupyter notebooks, but still miss a lot of common tasks, shortcuts, extensions. Would be great to be able to opt for jupyter or jupyter hub
    also see:
    http://feedback.databricks.com/forums/263785-product-feedback/suggestions/10703517-databricks-could-use-jupyter-as-a-front-end

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  13. Connection to Azure Data Lake Analytics Tables/views

    We have business data objects (tables / views) created on ADLA and we want to call them from Databricks can we have a connection same as ADLS.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. Data Bricks should be able to be integrated with Service Now to analyze past product problems

    DataBricks can be integrated to streaming data and product master data to get the master data as well as performance data of the device. But it should be able to integrate with Service now to query and auto suggest problems analyzing the telemetry of the device to suggest that there is a problem and suggest solution from past incidents.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  15. Provide support for mrsdeploy on Azure databricks

    I wish to deploy R model to Azure ML server using mrsdeploy from Azure databricks

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. Tags should be inherited from the deployment resource group

    We've been trying to utilize tags in our environment to identify resources for billing purposes. We've put a policy in place that forces new resources to inherit this tag from its resource group if it wasn't given one explicitly. Most Azure resources are handled fine, but we've discovered that the tags don't get inherited by the resources created by databricks in the managed resource group. Instead, we have to individually assign the tags to each Databricks cluster. I'd like to see new clusters that inherit the tags of either the Managed Resource Group or the Databricks service itself.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  17. job creation automation

    Is there any way to create ADB job in automated way. Currently I have to create jobs manually in all the environments.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  18. Can we access Event hub in Databricks using service principal

    Event hub document shows that we can access it using service principal and also I see that, we can access event hub using service principal using python libraries.

    Can we access Event hub in Databricks using service principal instead of SAS

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  19. Security level when connecting Databricks to other Azure services

    Hello

    the [following page](https://docs.microsoft.com/fr-fr/azure/databricks/administration-guide/cloud-configurations/azure/vnet-inject) states :

    "VNet injection, enabling you to:
    Connect Azure Databricks to other Azure services (such as Azure Storage) in a more secure manner using service endpoints."

    Data Factory is now a 'Trusted Service' in Azure Storage and Azure Key Vault firewall, we can connect to those services as ‘Trusted Service’ using the Data Factory managed identity and the firewall settings ‘Allow trusted Microsoft Services…’.

    Could you please explain why using "service endpoints is more secure" that using 'Trusted Service'?

    Reference : [Data Factory is now a 'Trusted Service' in Azure Storage and Azure Key…

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  20. Disabling 'import library' options

    I want to disable all possible ways of installing libraries that covers init script, UI, REST api, and condo

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Azure Databricks

Categories

Feedback and Knowledge Base