Azure Databricks

Azure Databricks is an Apache Spark-based analytics platform optimized for the Microsoft Azure cloud services platform. Designed with the founders of Apache Spark, Databricks is integrated with Azure to provide one-click setup, streamlined workflows, and an interactive workspace that enables collaboration between data scientists, data engineers, and business analysts.

We would love to hear any feedback you have for Azure Databricks.
For more details about Azure Databricks, try our documentation page.

  1. Multiple Cron schedules for one job

    Many simple schedules cannot be configured for a single job in Databricks due to the limitations of cron schedules. E.g. running a job every 40 minutes. Multiple schedules could provide such a frequency, but today, that would require having duplicate copies of the same job with different schedules.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  2. Enable Azure AD credential passthrough to ADLS Gen2 from PowerBI

    At present, the PowerBI connector uses token authentication. It would be ideal if this used AD auth, and that auth was passed down to the underlying source (Data lake gen2).

    This is currently only available within the workspace using High Concurrency clusters. But we would like non-technical users to use PowerBI

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  3. Unable to install azure-eventhub PYPI package

    Hi,

    Tried to install azure-eventhub PYPI package in databricks cluster but ended up in error

    Could not find a version that satisfies the requirement azure-eventhub (from versions: )
    No matching distribution found for azure-eventhub

    Does databricks have restrictions in installing libraries from PYPI? I could see the package in PYPI but why databricks is not able to find it

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. Add diagnostic logging configuration to ARM template

    It is possible to automatically add control plane diagnostic logging from Azure Databricks to Log Analytics. It is not possible to automate this process and must be done through the Portal. It would be great if this could be included in the ARM template or via Powershell.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  5. Support importing workspace libraries in the REST API

    Workspaces can contain notebooks, folders, and libraries. However, the REST API only support importing notebooks and .dac folders into a workspace. This feature request is to support importing libraries (e.g. .whl files) into a workspace.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. backquotes do not work as documented for SQL

    This documentation example, from: https://docs.microsoft.com/en-us/azure/databricks/getting-started/spark/dataframes#run-sql-queries

    select State Code, 2015 median sales price from data_geo

    Fails on

    SELECT CpuUtilization Average FROM t09c0e51a

    as does this:

    asparquet = asparquet.withColumnRenamed("CpuUtilization Average", "CpuUtilizationAverage")
    as_parquet.createOrReplaceTempView('t09c0e51a')
    sqlContext.sql("SELECT CpuUtilizationAverage FROM t09c0e51a").take(1)

    1
    asparquet = asparquet.withColumnRenamed("CpuUtilization Average", "CpuUtilizationAverage")
    2
    as_parquet.createOrReplaceTempView('t09c0e51a')
    3
    sqlContext.sql("SELECT CpuUtilizationAverage FROM t09c0e51a").take(1)

    org.apache.spark.sql.AnalysisException: Attribute name "CpuUtilization Average" contains invalid character(s) among " ,;{}()\n\t=". Please use alias to rename it.;

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  7. Tags should be inherited from the deployment resource group

    We've been trying to utilize tags in our environment to identify resources for billing purposes. We've put a policy in place that forces new resources to inherit this tag from its resource group if it wasn't given one explicitly. Most Azure resources are handled fine, but we've discovered that the tags don't get inherited by the resources created by databricks in the managed resource group. Instead, we have to individually assign the tags to each Databricks cluster. I'd like to see new clusters that inherit the tags of either the Managed Resource Group or the Databricks service itself.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  8. job creation automation

    Is there any way to create ADB job in automated way. Currently I have to create jobs manually in all the environments.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  9. Can we access Event hub in Databricks using service principal

    Event hub document shows that we can access it using service principal and also I see that, we can access event hub using service principal using python libraries.

    Can we access Event hub in Databricks using service principal instead of SAS

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  10. Security level when connecting Databricks to other Azure services

    Hello

    the [following page](https://docs.microsoft.com/fr-fr/azure/databricks/administration-guide/cloud-configurations/azure/vnet-inject) states :

    "VNet injection, enabling you to:
    Connect Azure Databricks to other Azure services (such as Azure Storage) in a more secure manner using service endpoints."

    Data Factory is now a 'Trusted Service' in Azure Storage and Azure Key Vault firewall, we can connect to those services as ‘Trusted Service’ using the Data Factory managed identity and the firewall settings ‘Allow trusted Microsoft Services…’.

    Could you please explain why using "service endpoints is more secure" that using 'Trusted Service'?

    Reference : [Data Factory is now a 'Trusted Service' in Azure Storage and Azure Key…

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. Disabling 'import library' options

    I want to disable all possible ways of installing libraries that covers init script, UI, REST api, and condo

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  12. Azure databricks cluster creation via Terraform

    I need help to deploy Azure Databricks cluster using terraform template.
    I have tried for some parameter like " azurermdatabricksworkspace" i can able to launch databricks ,but having issue on databricks cluster creation and not able to find any module /argument for same.
    Please help me to fix this .

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  13. python to Access directly using the storage account access

    The docs on https://docs.microsoft.com/en-us/azure/databricks/data/data-sources/azure/azure-datalake-gen2 show an example of using a storage account access key in scala - is there an example in python?

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. Azure Databrick Resource blocks RBAC permission assignment for the storage account at Management Group Level

    There is no way to deploy an Azure Databrick Resource and have RBAC permission assignment for the storage account at Management Group Level on the same subscription as of now.
    Can we have any temporary workaround for the same?

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  15. Can i call a java file in ADF

    Can i call a java file in ADF. Please provide proper inputs

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. Allow Databricks to deploy ml models from h2o java objects in shiny apps.

    A shiny .app file and a java object of the ML model from H2O are uploaded and a shiny server instance is deployed.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  17. Julia, need some advice from microsoft

    I am part of ExxonMobil and we start migrating apps to Azure, we have on an APP 13 Petabytes of Data , what is you suggestion for migration some of the AZURE groups suggesting DATA BOX, some other Express Routes we are trying an F5, what will be MS suggestion or who we can talk to, also what type of bandwidth is suggested, The reason I am asking is because I work on an AWS project and we migrate some data using a SNOWBALL equivalent of a DATA BOX and the transfer rate was very poor, any suggestion will be…

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  18. Job Run on specific days

    Currently Job settings are to run jobs weekly where we can't set the "days" in the week we want to run them.
    It takes all the days as default. Or can only select one day per Job.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  19. 1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  20. Fix Data Box Edge Configuration issue

    Hi,

    We have been working on configuring compute on the Data box edge and getting the below error. Please let us know how to resolve the issue. Thanks.

    Configuring Edge compute on dbe1

    message: (Http status code: 400) Could not create or update IoT role on 'dbe1'. An error occurred with the error code {NO_PARAM}. For more information, refer to the error code details (http://aka.ms/dbe-error-codes). If the error persists, contact Microsoft Support.

    I go to the url and I get this.

    <Error>

    <Code>BlobNotFound</Code>

    <Message>

    The specified blob does not exist. RequestId:01fe69a5-201e-0130-37f8-31b4b4000000 Time:2019-07-03T23:37:19.1084096Z

    </Message>

    </Error>

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Azure Databricks

Categories

Feedback and Knowledge Base