Data Lake

You can use this set to communicate with the Azure Data Lake team. We are eager to hear your ideas, suggestions, or any other feedback that would help us improve the service to bet fit your needs.

If you have technical questions, please visit our forums.
If you are looking for tutorials and documentation, please visit http://aka.ms/AzureDataLake.

  1. Tool like Azure Storage Explorer but for Data Lake

    Please post a supported or sample tool that provides a GUI and lets you upload and download local files to Azure Data Lake Store. It should be able to authenticate with AAD then display folders and let you upload and download to local files. It should not require any Azure subscription permissions, only WebHDFS permissions.

    I would consider this to be the equivalent of the Azure Storage Explorer... but for Azure Data Lake Store.

    199 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  13 comments  ·  Flag idea as inappropriate…  ·  Admin →
  2. Support interactive workloads within ADL Analytics

    Today, U-SQL only supports outputting data into files through batch execution. Enable interactive query capability that delivers results back to the client and w/o landing it into ADL. In addition, tools such as Tableau, PowerBI and other BI tools can connect and issue different queries while interactive ADLA performs necessary computation and delivers results of such computation back to the reporting and BI tools.

    181 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  14 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. USQL String Data Type has a size limit of 128KB

    USQL String Column Data Type has a size limit of 128KB. This limits uploading/processing the text data larger than 128kb through USQL job. For example, if the text data type in SQL has XML content, which size greater than 300KB, it fails uploading/processing with USQL. Can we increase the string data type size?

    169 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  14 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. Output to PowerBI

    Output to PowerBI

    157 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    20 comments  ·  Flag idea as inappropriate…  ·  Admin →
  5. JSON support for data analytics in azure portal

    data analysis for JSON data hosted in data lake store need to be processed by data analytics jobs directly and easily. Lot's of customers waiting for that. There is NO clear guidance on how to use JSON data using U-SQL query, and we might be loosing lots of business as most of the customers now a days have complex JSON data to be queried.

    114 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    7 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. Support output to SQL and SQL DW

    Today, ADLA supports reading from SQL and SQL DW, so if I'm operating on data in these sources, I would ideally prefer to write in the same data source as well to avoid unnecessary movement and silo-ing of data across multiple sources.

    94 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    13 comments  ·  Flag idea as inappropriate…  ·  Admin →
  7. 92 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  17 comments  ·  Flag idea as inappropriate…  ·  Admin →
  8. Add support for ADLS Gen2 to ADLA

    Would like the ability to use the ADL Analytics job-as-a-service to work with data in ADLS Gen2.

    77 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    12 comments  ·  Flag idea as inappropriate…  ·  Admin →
  9. 70 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    3 comments  ·  Flag idea as inappropriate…  ·  Admin →
    planned  ·  Saveen responded

    This work necessary to support this is in progress.

  10. Geo redundant storage / Disaster Recovery for Azure Data Lake Store

    Ability to store data in Azure Data Lake Store in on region and have that automatically be replicated to a different region. This is typically useful for disaster recovery scenarios.

    66 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    6 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. trigger azure functions based on files received in azure data lake store

    It does not seem to be possible to trigger azure functions based on files received in azure datalake store. in blob storage this works like a charm, but for ADLS this isn't possible. So my proposal is improve integration azure functions - azure data lake store: ability to trigger azure functions based on received files in azure data lake store.

    58 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  12. Support Expiration of files in ADLS

    In cosmos it is possible to set an expiration date when outputting a stream.

    I noticed ADLS recently started supporting expiration date in files.

    Please add to U-SQL the capability to set the expiration date (or for how long the file should exist before it expires; this would be even better) when outputting a recordset into a file in ADLS.

    55 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    3 comments  ·  Flag idea as inappropriate…  ·  Admin →
  13. Release ADL to Canadian region

    HDInsight has recently (Feb 2017) been made generally availalble in Canada. Nice to have ADL made available really soon. At least in preview. Any plans to share at this time?

    53 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    12 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. Excel Extractor

    Extractor that pulls from Excel Worksheets in a Workbook!

    48 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  7 comments  ·  Flag idea as inappropriate…  ·  Admin →
  15. Enable code/U-SQL to read header row

    provide functionality to read column row/schema of file dynamically

    47 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  3 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. SSIS Component - Azure Data Lake Store Source - Please include Text Qualifier option to load the files

    Currently "SSIS Component - Azure Data Lake Store Source" doesn't have option of Text Qualifier so while reading the file from ADL where the values are enclosed with a Text Qualifier like " couldn't be loaded.

    It will be great to have Text Qualifier option to load such kind of files directly from ADL store.

    43 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    3 comments  ·  Flag idea as inappropriate…  ·  Admin →
  17. Dynamic partition creation

    We currently have to manually create vertical partition buckets. This is fine when we’re creating a fixed number of partition buckets and know what they are upfront. However, when dealing with data with an unknown number of partition buckets, this poses a much bigger challenge and requires some external automation (e.g. a PowerShell script) to add new partitions ahead of inserting data that doesn’t belong to any existing partition bucket.

    It would be great if ADL/U-SQL had an option to dynamically create/add new partition buckets so that we’re not forced to ignore or move data to an ‘unknown’ catchall bucket…

    43 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  18. Native support for F# with USQL

    It would be great to see some F# support for USQL. I understand that you can already do UDFs in any .NET language (provided you inherit from the required base class or interface etc.) but having F# inline with USQL would be excellent. F#'s lightweight syntax and expression-based syntax would be a natural fit with the SQL section of USQL, and I think would provide a more seamless experience switching when between SQL and .NET code than SQL and C#.

    Pushing this further one could envisage an F# / USQL type provider along the lines of the SQL Client one…

    42 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  19. Allow rowset variables to be converted to scalar values in USQL

    I'm looking to make an output path string out of dates that are in an event-level table, so that the aggregate results can be saved and partitioned into paths according to their date.

    It appears that when I'm selecting a date from a rowset, it cannot be converted into a scalar string and used as an output path..

    40 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    7 comments  ·  Flag idea as inappropriate…  ·  Admin →
  20. Support to Update & Delete operations are in USQL

    Support to Update & Delete operations are in USQL; This great feature for Big data environments.

    40 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    3 comments  ·  Flag idea as inappropriate…  ·  Admin →
← Previous 1 3 4 5 16 17
  • Don't see your idea?

Feedback and Knowledge Base