Azure Synapse Analytics
-
[In Preview] MERGE statement support
Support the MERGE sql statement to support management of SCD1 / SCD2
512 votesThis feature is in private preview and will be available to all users in coming months.
-
Add support for "Delta Lake" file format in Azure Data Lake Store / HDFS
Today we can query data stored in parquet files on ADLS. It would be fantastic to extend this to support the new "Delta Lake" file format recently open-sourced by the DataBricks team ( see https://delta.io )
This would allow us to take advantage of ACID guarantees that the delta format brings to the data lake.
448 votesDelta Lake public preview is started in serverless SQL pools. Documentation is available here: https://docs.microsoft.com/en-us/azure/synapse-analytics/sql/query-delta-lake-format
-
User Defined Retention Periods for Restore Points
SQL DW doesn't have the capability to specify longer term retention capabilities as released for SQL DB recently. The workaround is to restore and pause. It would be great to have this capability for compliance and auditing requirements.
325 votesThank you for voting for this feature! We are aware of this scenario and are looking into ways of supporting this. In the meantime, stay tuned for an update and please continue voting for this feature.
-
Cross Database Querying
Enable elastic query for SQL DW - this would allow cross database query in SQL DataWarehouse.
271 votes -
Support for Recursive CTE
It would be great if you can add support for recursive CTEs for computing hierarchical data
203 votes -
Allow DWU to scale without interruption
The scaling disrupts the current session, and it's often not allowable in the production. Instead, it should finish off the existing session (sqls), and scale after that. That way, the scaling up or down can be done without causing the production disruption.
201 votesThank you for voting for this feature! We are aware of this scenario and are looking into ways of supporting this. In the meantime, stay tuned for an update and please continue voting for this feature.
-
Enable support for cross subscription restore
The current workaround for cross subscription restore is:
- Restore to a new logical server in the same prod subscription
- ‘Move’ the new logical server with the restored data warehouse to new subscription
It would be faster and simpler to enable restore directly to a different subscription.
196 votes -
Support for Entity Framework
Support for Entity Framework in Azure SQL Data Warehouse would allow for greater usability in general, but it would also allow for better support of other frameworks/protocols. Specifically, we have a need to serve our data through an OData feed, and Entity Framework's deferred execution would allow us to handle OData's Query Options in a much simpler way.
188 votesThank you for all the feedback folks. Unfortunately this is taking longer than we’d like. We will reach back out when it is on the roadmap and can share when we have an update.
-
Support for spatial data type
Spatial data shlould be supported by SQL Data Warehouse the same way it is supported by SQL Server.
185 votes -
[In preview] Rename a column name in SQLDW
This request is currently in Preview for columns in user tables.
See release notes:
https://docs.microsoft.com/en-us/azure/synapse-analytics/sql-data-warehouse/release-notes-10-0-10106-0#dec-2020
See sp_rename:
https://docs.microsoft.com/en-us/sql/relational-databases/system-stored-procedures/sp-rename-transact-sql?view=sqlallproducts-allversionsScenario: We want to change the column name as it is changed in the source. And in table where we are trying to do has more than 20Billion records.
Workaround: Since rename of column is not possible currently in SQL DW we are doing CTAS of that big table.
Issues With Work around: It is very slow because we can’t do CTAS on entire big table in single transaction because of transaction limits and we end up doing in CTAS and with…176 votesThis feature is in Preview for columns in user tables.
-
Polybase: PARTITIONED BY functionality when creating external tables.
I have a large number of files which are read with Hive using a partitioning scheme. PARTITIONED BY functionality, which is so commonly used in HIVE is missing from polybase.
159 votesThank you for your request. I would like to understand if this request concerns SQL DW or SQL Server implementation of PolyBase?
-
Expose File Properties as additional columns to load in SQL DW
It would be helpful if some of the properties of a BLOB file could be exposed in the external table such as: Last Modified, Filename, Size. It should be optional and could be defined in the EXTERNAL FILE FORMAT definition. Each Property should be appended to the end of the result set of an external table in the order given in the definition.
For instance, if we are loading a directory of several files we can read the filename and derive a new column. For example, we might have a directory with 20160504.csv and 20160505.csv and the contents of the…
156 votesThanks for your suggestion. We are looking into this scenario for a future release. 7633537
-
Delta lake support to SQL on Demand
Currently, Delta lake table is not supported to be queried using SQL on Demand. Adding this functionality would help to greatly use SQL on-demand for analyzing the delta lake table.
This will save time to query any Spark database delta lake table without spinning spark pools.
152 votesDelta Lake public preview is started in serverless SQL pools. Documentation is available here: https://docs.microsoft.com/en-us/azure/synapse-analytics/sql/query-delta-lake-format
-
Lower scale points for Dev/Test environments
SQL Data Warehouse scale stepping should be much smaller (10 DWU or even 1 DWU), than 100 DWU!
Please fix that ASAP!
Thank you!152 votesThank you for all the feedback folks. We are currently evaluating options for lower click stops for SQL Data Warehouse. Please continue to vote and comment on your scenario below. Thank you for your patience.
-
@@ROWCOUNT Support
Please enable @@ROWCOUNT?
Furthermore considering that we cannot switch NOCOUNT on, then why should it be such a secret/coded mission to track the number of rows affected by the most recent snippet of code? The proposed work-around seems onerous and doesn't always work (for dynamic SQL) yet my query window still knows and always-prints the fact that it returned 15 rows to me, for example, so why can we not leverage this elementary information inside the selfsame query (?):
SELECT SUM(rowcount) AS rowcount
FROM sys.dmpdwsqlrequests
WHERE rowcount <> -1
AND request_id IN
…( SELECT140 votes -
[In Preview] Hash distribute fact table on multiple columns
A large fact table often has multiple join keys to multiple dimension table, so while it can be opitmised for one join it always creates data movement for other joins. The hash distribution should accept multiple columns so all joins are optimised and create no data movement for dimension joins.
130 votesThis feature is currently in preview and will be available to all users in coming months.
-
Temporal Table support in AzureSQLDW
Is this on the roadmap? Currently they are not a thing in the Parallel DW:
https://msdn.microsoft.com/en-us/library/dn935015.aspx?f=255&MSPPError=-2147217396124 votesThanks for your suggestion. We are looking into this scenario for a future release. 10714515
-
offer a PaaS version of Master Data Services (MDS)
Master Data Services is vitally important for data governance and enterprise data warehousing. Microsoft has a great PaaS data warehouse offer. But for MDS, customers must deploy SQL Server Enterprise on a VM.
117 votes -
Provide File Format to read and load JSON and XML data from Storage
Able to read the JSON and XML file as part of external system OR provide functionality like Extractor in Data Lake analytics so we can write our own Extractor to read respective data format.
113 votes -
111 votes
- Don't see your idea?