Delta Lake format provides ability to run time-travel queries. This should be added to the existing Delta Lake support in Synapse pools.35 votes
Would love to have support for the language R in the notebook experience of Synapse Studio.75 votes
Currently, Delta lake table is not supported to be queried using SQL on Demand. Adding this functionality would help to greatly use SQL on-demand for analyzing the delta lake table.
This will save time to query any Spark database delta lake table without spinning spark pools.156 votes
Please add the OPENROWSET functionality dedicated pools.
1. More (and faster) parser options over External File Format such as row delimiter
2. Can auto-infer schema
3. More convenient to define the file format directly
4. syntactical harmony between serverless and dedicated.61 votes
Is it possible to add the use of global parameters in Synaps management studio? Exactly the same as in Azure data Factory. This makes life so much easier when working with differente Environments.45 votes
The "Common Data Model" (CDM) format is becoming increasingly popular. Therefore it would be important that this connector not only exists in the ADF, but that it is also possible to read and write (via CETAS) in CDM directly from SQL on-demand.
SELECT * FROM
BULK STORAGE ACCOUNT,
FORMAT = CDM
) AS [r];
CREATE EXTERNAL TABLE cdm.cdmtable
LOCATION = pathtofolder,
DATASOURCE = externaldatasourcename
FILEFORMAT = CDM
) AS SELECT * FROM source61 votes
It would be nice to be able to manage deployments of serverless sql databases through SSDT and Azure DevOps Pipelines as you would a dedicated pool or an Azure SQL database.15 votes
https://cloudblogs.microsoft.com/sqlserver/2019/11/07/new-in-azure-synapse-analytics-cicd-for-sql-analytics-using-sql-server-data-tools/ describes how SQL Server Data Tools support for Synapse not mentioning that this is only for Dedicated Pools. We need similar support for Synapse Serverless as well otherwise CI/CD will be hard to handle.26 votes
Today we can query data stored in parquet files on ADLS. It would be fantastic to extend this to support the new "Delta Lake" file format recently open-sourced by the DataBricks team ( see https://delta.io )
This would allow us to take advantage of ACID guarantees that the delta format brings to the data lake.464 votes
Synapse had git integration, but when we need to save / commit our work of notebook, we can only get JSON definition file of notebook in git repo.
Currently If we need to make ipynb source code, we can only use "Export" btn to generate ipynb file, but its not integrated with git repo, so it still lose source control ability of that ipynb file, also its hard to code review using JSON format file.
It would be very helpful if we can also keep ipynb format when integrating with Git Repo7 votes
Enable a way to refer and add a custom .jars when initiating a Synapse Spark cluster to enable Synapse notebooks to import from the .jars40 votes
Request to shorten execution time on pipeline for more than 2 notebooks on Synapse Studio.
From my observation, the reason why it takes time to run each notebook might be each needs to establish Spark session that takes time.
Seeing parameters on Spark pool, there are not useful parameters to achieve the requirement.
If same one spark pool is being used on pipeline across multiple Notebooks, please consider same session can be used to shorten total execution time.16 votes
Please integrate Azure Analysis services development for building semantic models in Azure Synapse Analytics
We are very pleased to develop solutions in ASA. For several customers we build Azure Analysis services Models. It would be great, if we don'T have to leave the ASA Studio. Thx10 votes
As of now, it is not possible to reuse an existing Integration Runtime used in a Data Factory with Synapse Studio. This makes the migration and testing process much more complicated than it should be.53 votes
For users coming from a DataBricks or Jupyter environment to Synapse, one of the features that Synapse does not have is auto-saving a notebook. So if I forget to click the "Publish" button, and happen to close my browser or my browser crashes/closes unexpectedly, I lose all of my work. Where with ADB or AML Notebooks, I don't have to worry about that extra step.6 votes
Currently we are getting authentication errors when using Private nuget feeds.
Unable to load the service index for source "json path for our Nuget feed" [/tmp/nuget/5730--f405c3cb-1015-48fa-ac09-a29344c71270/Project.fsproj]
/usr/share/dotnet/sdk/3.1.409/NuGet.targets(128,5): error : Response status code does not indicate success: 401 (Unauthorized). [/tmp/nuget/5730--f405c3cb-1015-48fa-ac09-a29344c71270/Project.fsproj]
Please add nuget feed authentication support for using nuget packages for private libraries
" #r "nuget:Microsoft.Spark, 1.0.0" ```5 votes
Parquet number of files, file size, and parquet row group size greatly influence query speeds of Synapse serverless. Yet parquet file creation through CETAS cannot be configured in any way except for the type of compression. Moreover CETAS is not consistent in parquet file creation; the generated parquet file size and the number of files created varies wildly. I've seen CETAS queries return a single 1.5GB file, or dozens of 1MB files. Given this behavior, it is very hard to use CETAS as part of a production data pipeline, at the moment it is more of a prototyping tool.16 votes
We do plan to improve CETAS when it comes to partitioning the output and balancing the size of files.
We didn’t plan to allow specifying number of files, we are interested to see why do you need that?
Currently Spark Core Quotas cannot be viewed by the customer in the Azure Portal. The only way to know a quota has been reached is by errors that result when the Spark job fails. Spark Core Quotas should be viewable in the Azure Portal alongside all other resource quotas.
Additionally, quotas are currently per workspace. It's a significant hassle to have to request quota increases per workspace. Please make Spark Core Quotas per subscription.5 votes
The ability to run EXPLAIN PLAN syntax for serverless SQL pool queries Azure Synapse Analytics.12 votes
- Don't see your idea?