Azure Synapse Analytics

We would love to hear your ideas for new features for Azure Synapse Analytics. Below, enter a new idea or upvote an existing one. The Synapse engineering team pays attention to all requests.

If instead you need a technical question answered or help, try the these options: DocumentationMSDN forum, and StackOverflow. If you need support, please open a support ticket.

  • Hot ideas
  • Top ideas
  • New ideas
  • My feedback
  1. PolyBase: allow skipping header rows in delimited text files

    Setting reject rows in Polybase to ignore the header row is a hack. And if you have all string columns your header won't be rejected. Please add an IGNOREHEADERROWS = <numRows> option to Polybase

    175 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    17 comments  ·  SQL/Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  2. PolyBase: support for Azure Data Lake Store

    Polybase should add support for Azure Data Lake. I'm hopeful that will give us an option to do Polybase on top of files which are encrypted at rest.

    143 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    10 comments  ·  SQL/Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  3. PolyBase: allow retrieving rejected rows in an error file

    Let's say you define an external table with:

        REJECT_VALUE = 1000,
    
    REJECT_TYPE = value

    That means that it can reject and discard 1000 rows without failing the entire command.

    Please enhance Polybase to give us visibility into which rows were rejected and why. I would propose you can define an error output file where rejected rows get written. Either that or let us query some DMV to get a copy of the complete row as a string and the error reason.

    112 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    14 comments  ·  SQL/Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  4. PolyBase: support for encrypted Azure Blob Storage (Storage Service Encryption)

    There is a requirement to never leave unencrypted data anywhere. Inbound data files are encrypted but will have to be decrypted before Polybase can read them. In order to satisfy this requirement, Polybase will have to have ability to read encrypted files.

    28 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    6 comments  ·  SQL/Polybase  ·  Flag idea as inappropriate…  ·  Admin →

    Starting April 2016, Azure Storage Service Encryption (SSE) for Azure Blob Storage is in Public Preview. With this feature, Azure Storage automatically encrypts your data prior to persisting to blob storage and decrypts prior to retrieval, providing encryption at rest. We have tested this feature and PolyBase supports reading/writing data to encryption-enabled Azure Blob Storage. To learn more about Azure Storage Service Encryption, refer to https://azure.microsoft.com/en-us/documentation/articles/storage-service-encryption/.

    Thanks again for sharing your feedback with us. 6891055

  5. Enable Integrity and Privacy options for hadoop.rpc.protection setting

    Enable Integrity and Privacy options for hadoop.rpc.protection setting.
    Currently it only connects to Cloudera when set to Authentication, this doesn't provide sufficient security for storing/processing sensitive data.

    MS Work Items: 9519053 and 10001755.

    25 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    5 comments  ·  SQL/Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  6. Polybase to treat NULL string as Null values and not text.

    When we have files generated by third party we have little control over the file format, in our case huge data files with lots of NULL value are causing us trouble.

    14 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    4 comments  ·  SQL/Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  7. varchar(max) with Polybase

    need support for varchar(max) with polybase

    14 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  SQL/Polybase  ·  Flag idea as inappropriate…  ·  Admin →

    This is currently a feature we support. We’ve recently increased the row support size to 1MB now. Generally one should avoid unnecessarily specifying a column to be max width if not necessary for performance reasons as well.

  8. Add Polybase Rejected Rows DMV

    The REJECTTYPE and REJECTVALUE settings on a Polybase External Table are very useful for managing invalid data types, but they are very limited if you're using Polybase as a data loading mechanism. In order for this to be useful, we need visibility of which rows were rejected, even if it was within the tolerance.

    I would suggest a generic system view that logged the failed row as an untyped string (or JSON/XML string if in the SQL engine rather than the Azure SQL Datawarehouse engine).

    This would read something like:

    rejectid, objectid, querydate, rejectstring

    8 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  SQL/Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  9. Polybase: Remove/increase row size from 32k

    Polybase is currently limited to a maximum row size of 32k. Request that this be increased by orders of magnitude.

    7 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  SQL/Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  10. Support defining external data source as a public blob

    We would like Polybase to support a public blob as an external data source. This should not require any credentials. The benefit is that for public datasets already stored on a Azure public blob, I can directly reference it in SQL DW and import that into the data warehouse without making another copy to my own private blob.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  SQL/Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  11. sql server 2016 polybase data lake store

    On prem Polybase should support Azure Data Lake Store and push down query to Azure Data Lake Analytics

    0 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  SQL/Polybase  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Feedback and Knowledge Base