Azure Databricks

Azure Databricks is an Apache Spark-based analytics platform optimized for the Microsoft Azure cloud services platform. Designed with the founders of Apache Spark, Databricks is integrated with Azure to provide one-click setup, streamlined workflows, and an interactive workspace that enables collaboration between data scientists, data engineers, and business analysts.

We would love to hear any feedback you have for Azure Databricks.
For more details about Azure Databricks, try our documentation page.

  1. Personal Access Token Security Improvements

    Most Databricks users end up needing to generate a Personal Access Token - which I am guessing is why Microsoft started to default that setting to ON.

    The problem is, from an Access Control perspective these tokens present a massive risk to any organization because there are no controls around them.

    These tokens allow direct access to everything the user has access to and all it takes to cause a major data breach is for one user to accidentally post one of these tokens on a public forum or GitHub.

    Here are a few specific issues:
    1. Even though conditional…

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  2. ARM templet with databricks within vnet can only be deployed once

    Cannot redeploy arm template with databricks with vnet integration. More details here: https://github.com/Azure/azure-quickstart-templates/issues/6670

    Actually I get the same error when issueing az network nsg create command for already existing nsg (used by databricks).

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. Azure databricks cluster creation via Terraform

    I need help to deploy Azure Databricks cluster using terraform template.
    I have tried for some parameter like " azurermdatabricksworkspace" i can able to launch databricks ,but having issue on databricks cluster creation and not able to find any module /argument for same.
    Please help me to fix this .

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. python to Access directly using the storage account access

    The docs on https://docs.microsoft.com/en-us/azure/databricks/data/data-sources/azure/azure-datalake-gen2 show an example of using a storage account access key in scala - is there an example in python?

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  5. NCv3 in North Europe

    Tesla K80 cards are super slow and lack mixed precision support, furthermore, they're incompatible with Nvidia RAPIDS, because they're very old.
    Google doesn't even give away K80 for free in colab unless all P100's are taken.
    Having only K80s available in north europe is outrageos.
    We've had to shuffle data and models back and forth between north and west to train models in west. This is becoming very cumbersome and wasting a lot of time for us.
    Please add contemporary GPUs in North Europe

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. Support for enqueuing job clusters onto an instance pool

    Using instance pools to optimize the runtime of smaller, automated jobs is currently at odds with the built-in scheduling system.

    This is because an instance pool will simply reject a job for which it can't immediately procure the required amount of nodes.

    This is a proposal to have an enqueuing behavior such that an automated job will instead wait (possibly with a configurable upper time limit) for resources to become available. The requirement would then be that either the minimum autoscaling configuration or the fixed number of nodes would be satisified at the time of job start.

    Having this functionality…

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Strong Feedback  ·  Flag idea as inappropriate…  ·  Admin →
  7. Azure Databrick Resource blocks RBAC permission assignment for the storage account at Management Group Level

    There is no way to deploy an Azure Databrick Resource and have RBAC permission assignment for the storage account at Management Group Level on the same subscription as of now.
    Can we have any temporary workaround for the same?

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  8. Can i call a java file in ADF

    Can i call a java file in ADF. Please provide proper inputs

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  9. Allow Databricks to deploy ml models from h2o java objects in shiny apps.

    A shiny .app file and a java object of the ML model from H2O are uploaded and a shiny server instance is deployed.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  10. Julia, need some advice from microsoft

    I am part of ExxonMobil and we start migrating apps to Azure, we have on an APP 13 Petabytes of Data , what is you suggestion for migration some of the AZURE groups suggesting DATA BOX, some other Express Routes we are trying an F5, what will be MS suggestion or who we can talk to, also what type of bandwidth is suggested, The reason I am asking is because I work on an AWS project and we migrate some data using a SNOWBALL equivalent of a DATA BOX and the transfer rate was very poor, any suggestion will be…

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. Job Run on specific days

    Currently Job settings are to run jobs weekly where we can't set the "days" in the week we want to run them.
    It takes all the days as default. Or can only select one day per Job.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  12. 1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  13. Fix Data Box Edge Configuration issue

    Hi,

    We have been working on configuring compute on the Data box edge and getting the below error. Please let us know how to resolve the issue. Thanks.

    Configuring Edge compute on dbe1

    message: (Http status code: 400) Could not create or update IoT role on 'dbe1'. An error occurred with the error code {NO_PARAM}. For more information, refer to the error code details (http://aka.ms/dbe-error-codes). If the error persists, contact Microsoft Support.

    I go to the url and I get this.

    <Error>

    <Code>BlobNotFound</Code>

    <Message>

    The specified blob does not exist. RequestId:01fe69a5-201e-0130-37f8-31b4b4000000 Time:2019-07-03T23:37:19.1084096Z

    </Message>

    </Error>

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. Databricks VNET Injection doesn't work

    When using Azure Databricks with VNET injection, whether you deploy from the portal or deploy via an Arm Template with VNET Injection and specify the VNET, subnets, etc, it creates a new VNET either way, it won't use your existing vnet and subnets. This is frustrating.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  15. databricks

    our Databricks test environment is always being charged even with the promise of having 15 days at no cost

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. Azure databricks Spark logs shows recent logs sorted by Date

    Azure databricks shows spark logs 1 month old as the most recent. Difficult to find the date of failed job and then analyse.
    They are not even sorted by size.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  17. Provide guidance on uiDefinitionUri in the documentation

    There is no information on how to use this option in the ARM template of Databricks. Some guidance would be nice.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  18. Run Jar like just other programs but not library for reuse

    I have algorithms written in jar and compiled as JAR.
    I would like to submit them on ML cluster not as a library but just another program(notebook scripts). Upgraded/modified Jar would need not then require clean up of already installed older version from the cluster.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  19. Impala as hosted MPP platform

    We need a more cost effective solution for MPP sql query engine for processing large volume of data generated. Azure DW is competitive offering and is very good but really expensive.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  20. Data Center Rollout

    Is there a plan to expand Azure Databricks into more data centers? There's a lot of interest for it in Canada but I haven't seen any announcements.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Azure Databricks

Categories

Feedback and Knowledge Base