SQL Data Warehouse

Do you have an idea or suggestion based on your experience with SQL Data Warehouse? We would love to hear it! Please take a few minutes to submit your idea or vote up an idea submitted by another SQL Data Warehouse customer. All of the feedback you share in these forums will be monitored and reviewed by the SQL Data Warehouse engineering team. By suggesting or voting for ideas here, you will also be one of the first to know when we begin work on your feature requests and when we release the feature.

Remember that this site is only for feature suggestions and ideas! If you have technical questions please try our documentation, MSDN forum, or StackOverflow. If you need support, please open a support ticket with us.

  • Hot ideas
  • Top ideas
  • New ideas
  • My feedback
  1. Polybase: PARTITIONED BY functionality when creating external tables.

    I have a large number of files which are read with Hive using a partitioning scheme. PARTITIONED BY functionality, which is so commonly used in HIVE is missing from polybase.

    75 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    10 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  2. Enable SQL DW as data source for external tables in SQL DB

    Querying SQL DW from SQL DB via external tables currently does not work due to an implicit SET LANGUAGE statement issued by the elastic queries component. It sets the Language to N'usenglish' (not supported) vs. usenglish (supported I guess). Same for SET NUMERICROUNDABORT, set to OFF (not supported). While you can explicitly set the NUMERICROUNDABORT to ON at run-time, you cannot do SET LANGUAGE us_english on SQL DB. Maybe other issuesd are to be found but this scenario is important.

    35 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    4 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  3. Polybase support for other cloud data sources

    we currently has Polybase to Azure Blob Store.

    we need Polybase to the following data stores:

    HDInsight with push down compute?
    SQL Azure with push down compute?
    DocumentDB with push down compute?
    Azure Data Lake Store with push down to Azure Data Lake Analytics?

    24 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  4. Support of StorageTable as DataSource

    Please add support of Azure TableStorage as PolyBase DataSource.
    To be honest I can't believe its not already available. :O

    21 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  5. Predicate pushdown for ADLS sources

    As Azure SQL DW now also supports ADLS, it would be great if we could also leverage the distributed processing capabilities of ADLS to push computation down to ADLS in a similar way as it works for HDFS/Hadoop already

    19 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    2 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  6. 18 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    2 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  7. ALTER EXTERNAL DATA SOURCE

    If a data source moves or changes then all external tables must be deleted, the data source deleted and recreated, then the external tables recreated.

    This may be experienced during a data warehouse migration, and during a disaster recovery.

    Support for ALTER EXTERNAL DATA SOURCE would make this a much more simple task.

    14 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    2 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  8. Support Append Blob

    Support for Azure storage Append Blob with PolyBase External tables

    14 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    4 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  9. Enable AD authentication for Polybase

    Currently SAS is the only option for Polybase to access blob storage or datawarehouse.

    13 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →

    Thanks for the request! Currently PolyBase only supports Storage Account Keys when connecting to Azure Storage Blobs. SAS token support is on our backlog.
    Today, Storage does not support AD as an authentication method.

    We will update when we move this feature through the backlog.

  10. Hive metastore integration for polybase / Azure SQL DW

    Hive metastore integration for polybase / Azure SQL DW
    I want to be able to seamless access/ import /join on tables that are already captured in my common hive metastore. This would significantly help integrating DW in our big data infrastructure and eliminate the huge duplicated maintenance effort of keeping external table definitions in sync.

    13 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  11. PolyBase:allow encoding in file format and polybase will take care of encoding.

    PolyBase: Allow encoding in file format and polybase will take care of encoding.

    8 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  12. polybase:allow insert new data to existing exteranl table

    right now after an external table has been created we can't add data to it, please update the product so that would be possible

    7 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  13. PolyBase: Support String Columns For Predicate Pushdown

    Predicate pushdown to Hadoop sources in SQL Server 2017 and 2019 CTP 2.2 works for date, time, and numeric data types (as the documentation at https://docs.microsoft.com/en-us/sql/relational-databases/polybase/polybase-pushdown-computation?view=sql-server-2017 correctly notes). It would be great if we could also push down string predicates from VARCHAR or NVARCHAR data types.

    Supporting LIKE filters would be nice, but even exact matches and support just for what's covered for numeric data types could help.

    6 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  14. Improve row size!

    Polybase (at least on SQL Server 2016) is limited to a maximum row size of 32k. Request to increase largely this value!

    6 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  15. Polybase old dates into ORC files fails

    The following CETAS statement fails. Apparently there's an issue with old dates and ORC files? Please fix.

    CREATE EXTERNAL FILE FORMAT ORC_Snappy
    WITH (

    FORMAT_TYPE = ORC  
    
    , DATA_COMPRESSION = 'org.apache.hadoop.io.compress.SnappyCodec'

    );

    create EXTERNAL TABLE dbo.testexternaltable WITH (DATASOURCE = myds, LOCATION = '/testexternaltable', FILEFORMAT = ORC_Snappy) AS
    select cast('1910-08-13' as date) as dt;

    6 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    3 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  16. Knox & Polybase

    The Apache Knox Gateway (“Knox”) provides perimeter security so that the enterprise can confidently extend Hadoop access to more of those new users while also maintaining compliance with enterprise security policies.

    More and more enterprises are using it as the only way to access to the cluster data.

    The security is more than never a point of attention so it would be great if Polybase could connect to a Hadoop cluster using the Knox Gateway.

    6 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  17. Option to disable polybase validation on external table creation scripts

    When External Tables creation scripts are executed without existing files to back them, the scripts fail. Provide an option to disable validation for automated deployments where the data does not yet exist however the schema and external tables should be created.

    Creation of objects should not have any data dependencies.

    Not having External Tables deployed causes subsequent script dependencies (views, etc) to fail.

    5 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  18. ORC file format should distinguish blank and null

    ORC files do not seem to be able to distinguish blank strings and nulls.

    5 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    2 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  19. Limit the size of ORC files produced by CREATE EXTERNAL TABLE AS SELECT command

    Recently I worked with Microsoft support on an issue using CREATE TABLE AS SELECT to import data from external tables using ORC file format. Most queries completed just fine, but some ran into errors like "Java heap space". Those queries were hitting external tables backed by huge ORC files (ranged from 17GB to 33GB per file). The product team confirmed there is a design limit that makes those files unqualified. I then split the files further down to less than 12GB per file (total data volume was kept the same) and the CTAS queries then ran just fine.
    So obviously…

    5 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  20. Export data (simple custom export) to a single file

    Enable exporting data to a single file with custom export options: file format, max size, custom null, customer escape character, encryption, compression, etc

    4 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Polybase  ·  Flag idea as inappropriate…  ·  Admin →
← Previous 1
  • Don't see your idea?

Feedback and Knowledge Base