Azure Databricks

Azure Databricks is an Apache Spark-based analytics platform optimized for the Microsoft Azure cloud services platform. Designed with the founders of Apache Spark, Databricks is integrated with Azure to provide one-click setup, streamlined workflows, and an interactive workspace that enables collaboration between data scientists, data engineers, and business analysts.

We would love to hear any feedback you have for Azure Databricks.
For more details about Azure Databricks, try our documentation page.

We welcome user feedback and feature requests!

(thinking…)

Enter your idea and we'll search to see if someone has already suggested it.

If a similar idea already exists, you can support and comment on it.

If it doesn't exist, you can post your idea so others can support it.

Enter your idea and we'll search to see if someone has already suggested it.

  1. Git Integration with VSTS

    Azure Databricks should support Git Integration with VSTS. Right now there is only Github and Bitbucket.

    26 votes
    Sign in
    Check!
    (thinking…)
    Reset
    or sign in with
    • facebook
    • google
      Password icon
      Signed in as (Sign out)

      We’ll send you updates on this idea

      3 comments  ·  Flag idea as inappropriate…  ·  Admin →
    • ssh access to databricks cluster VMs

      It should be possible to ssh into azure databricks cluster VMs. It it possible on Databricks on AWS

      https://docs.databricks.com/user-guide/clusters/ssh.html

      12 votes
      Sign in
      Check!
      (thinking…)
      Reset
      or sign in with
      • facebook
      • google
        Password icon
        Signed in as (Sign out)

        We’ll send you updates on this idea

        0 comments  ·  Flag idea as inappropriate…  ·  Admin →
      • support low-priority VMs

        Azure Databricks and low-priority VMs are a great match. Their use cases overlap heavily: https://docs.microsoft.com/en-us/azure/batch/batch-low-pri-vms#use-cases-for-low-priority-vms.

        10 votes
        Sign in
        Check!
        (thinking…)
        Reset
        or sign in with
        • facebook
        • google
          Password icon
          Signed in as (Sign out)

          We’ll send you updates on this idea

          1 comment  ·  Flag idea as inappropriate…  ·  Admin →
        • Allow custom/CIDR ranges for deploying Databrick in VNET

          Current ARM template and implementation of Databricks being deployed in VNET only allows /16 CIDR before deployment and does not allow manipulation to shrink /16 CIDR after deployment. This is a huge problem in larger enterprises where a /16 takes up 64k IPs and VNET technology only allows 8192 MAX Private IPs in a VNET. This presents a challenge for enterprises carving huge swaths of /16 space especially when WAN routing comes into place for the organization. The ability to define and deploy your own VNET with a /22 or other /CIDR range should be allowed.

          6 votes
          Sign in
          Check!
          (thinking…)
          Reset
          or sign in with
          • facebook
          • google
            Password icon
            Signed in as (Sign out)

            We’ll send you updates on this idea

            2 comments  ·  Flag idea as inappropriate…  ·  Admin →
          • support reading avro files from azure blob storage

            I use EventHub capture to storage account feature and want to load it's avro files from Azure Databricks, it works perfectly on dbfs, but when i try to load it directly from blob storage (based on this article https://docs.azuredatabricks.net/spark/latest/data-sources/azure/azure-storage.html) it fails with the following error:

            shaded.databricks.org.apache.hadoop.fs.azure.AzureException: shaded.databricks.org.apache.hadoop.fs.azure.AzureException: Container events in account pilfleetml.blob.core.windows.net not found, and we can't create it using anoynomous credentials, and no credentials found for them in the configuration.
            ---------------------------------------------------------------------------
            Py4JJavaError Traceback (most recent call last)
            <command-486363724788735> in <module>()
            ----> 1 avroDf = spark.read.format("com.databricks.spark.avro").load("wasbs://events@pilfleetml.blob.core.windows.net/pil-fleet-eh/pil-fleet-ml/0/2018/01/25/14/55/*")

            /databricks/spark/python/pyspark/sql/readwriter.py in load(self, path, format, schema, **options)
            157 self.options(**options)
            158 if…

            5 votes
            Sign in
            Check!
            (thinking…)
            Reset
            or sign in with
            • facebook
            • google
              Password icon
              Signed in as (Sign out)

              We’ll send you updates on this idea

              2 comments  ·  Flag idea as inappropriate…  ·  Admin →
            • 3 votes
              Sign in
              Check!
              (thinking…)
              Reset
              or sign in with
              • facebook
              • google
                Password icon
                Signed in as (Sign out)

                We’ll send you updates on this idea

                0 comments  ·  Flag idea as inappropriate…  ·  Admin →
              • Any update on HIPAA + HITRUST compliance with this offering?

                Our team works in healthcare and this could be something to look into if it meets all the compliance checks.

                3 votes
                Sign in
                Check!
                (thinking…)
                Reset
                or sign in with
                • facebook
                • google
                  Password icon
                  Signed in as (Sign out)

                  We’ll send you updates on this idea

                  1 comment  ·  Flag idea as inappropriate…  ·  Admin →
                • Integration with VSCode

                  Should be able to execute and deploy code to Azure Databricks clusters from VS Code

                  2 votes
                  Sign in
                  Check!
                  (thinking…)
                  Reset
                  or sign in with
                  • facebook
                  • google
                    Password icon
                    Signed in as (Sign out)

                    We’ll send you updates on this idea

                    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
                  • Stream Data into a Power BI Dataset

                    Similar to Azure Stream Analytics functionality, the ability to use Structured Streaming in Spark to stream/write data directly into a Power BI Dataset.

                    2 votes
                    Sign in
                    Check!
                    (thinking…)
                    Reset
                    or sign in with
                    • facebook
                    • google
                      Password icon
                      Signed in as (Sign out)

                      We’ll send you updates on this idea

                      0 comments  ·  Flag idea as inappropriate…  ·  Admin →
                    • Allow to pause service when not in use

                      Similar to SQL DW and VMs, can we get the option to pause the service when not in use to minimize costs?

                      2 votes
                      Sign in
                      Check!
                      (thinking…)
                      Reset
                      or sign in with
                      • facebook
                      • google
                        Password icon
                        Signed in as (Sign out)

                        We’ll send you updates on this idea

                        0 comments  ·  Flag idea as inappropriate…  ·  Admin →
                      • Apache Beam on Azure Databricks

                        Apache beam is an open source batch and streaming engine with unified model that runs on any execution engine, including Spark. It has powerful semantics that elegantly solves real world challenges in both streaming and batch processing. It recently got also some Scala based abstractions on top of it, which enables succinct and correct expressiveness of windowing, triggering, out of order events and further more. It also has been chosen from some successful cloud born companies that are challenged with vast amounts of data.

                        1 vote
                        Sign in
                        Check!
                        (thinking…)
                        Reset
                        or sign in with
                        • facebook
                        • google
                          Password icon
                          Signed in as (Sign out)

                          We’ll send you updates on this idea

                          0 comments  ·  Flag idea as inappropriate…  ·  Admin →
                        • Dark Theme for Azure Databricks Portal

                          add a dark theme option for the whole Azure Databricks portal for those of us who like working in the dark ;)

                          1 vote
                          Sign in
                          Check!
                          (thinking…)
                          Reset
                          or sign in with
                          • facebook
                          • google
                            Password icon
                            Signed in as (Sign out)

                            We’ll send you updates on this idea

                            0 comments  ·  Flag idea as inappropriate…  ·  Admin →
                          • Support conditional access

                            I really want to use conditional access in conjunction with Azure AD authentication to restrict the login locations. This does not seem possible today.

                            1 vote
                            Sign in
                            Check!
                            (thinking…)
                            Reset
                            or sign in with
                            • facebook
                            • google
                              Password icon
                              Signed in as (Sign out)

                              We’ll send you updates on this idea

                              0 comments  ·  Flag idea as inappropriate…  ·  Admin →
                            • Dbutils.fs.mount doesn't support mounting Azure China Storage, seems endpoint .core.windows.net was hardcoded

                              I created a Databricks env on Global Azure (portal.azure.com) and tried to access data stored in China (portal.azure.cn), when configured proper storage account name and key, I can access data in China Azure using dbutils.fs.ls. But trying to mount the storage account failed with below error. Apparently .core.windows.net was hardcoded somewhere.

                              dbutils.fs.mount(
                              source = "wasbs://mycontainer@<myaccount>.blob.core.chinacloudapi.cn/path",
                              mount_point = "/mnt/mymount",
                              extra_configs = {"fs.azure.account.key.<myaccount>.blob.core.chinacloudapi.cn": "ufU474A47XXXXXXXXXXXXXXXXXXXXXXXXXXZZU0LZmxQL5P1vLDH8FbGcdDGCVWX2cIGR"}
                              )

                              java.rmi.RemoteException: java.lang.IllegalArgumentException: Could not retrieve either the Azure storage account access key through option key fs.azure.account.key.<myaccount>.blob.core.chinacloudapi.cn.blob.core.windows.net or the SAS token for the Azure container through option key fs.azure.sas.mycontainer.<myaccount>.blob.core.chinacloudapi.cn.blob.core.windows.net.; nested exception is:

                              1 vote
                              Sign in
                              Check!
                              (thinking…)
                              Reset
                              or sign in with
                              • facebook
                              • google
                                Password icon
                                Signed in as (Sign out)

                                We’ll send you updates on this idea

                                0 comments  ·  Flag idea as inappropriate…  ·  Admin →
                              • Provide support for mrsdeploy on Azure databricks

                                I wish to deploy R model to Azure ML server using mrsdeploy from Azure databricks

                                1 vote
                                Sign in
                                Check!
                                (thinking…)
                                Reset
                                or sign in with
                                • facebook
                                • google
                                  Password icon
                                  Signed in as (Sign out)

                                  We’ll send you updates on this idea

                                  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
                                • Don't see your idea?

                                Azure Databricks

                                Feedback and Knowledge Base