Data Lake

You can use this set to communicate with the Azure Data Lake team. We are eager to hear your ideas, suggestions, or any other feedback that would help us improve the service to bet fit your needs.

If you have technical questions, please visit our forums.
If you are looking for tutorials and documentation, please visit http://aka.ms/AzureDataLake.

How can we improve Microsoft Azure Data Lake?

(thinking…)

Enter your idea and we'll search to see if someone has already suggested it.

If a similar idea already exists, you can support and comment on it.

If it doesn't exist, you can post your idea so others can support it.

Enter your idea and we'll search to see if someone has already suggested it.

  1. Release ADL to Canadian region

    HDInsight has recently (Feb 2017) been made generally availalble in Canada. Nice to have ADL made available really soon. At least in preview. Any plans to share at this time?

    53 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    12 comments  ·  Flag idea as inappropriate…  ·  Admin →
  2. 92 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  17 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. 28 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    3 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. USQL String Data Type has a size limit of 128KB

    USQL String Column Data Type has a size limit of 128KB. This limits uploading/processing the text data larger than 128kb through USQL job. For example, if the text data type in SQL has XML content, which size greater than 300KB, it fails uploading/processing with USQL. Can we increase the string data type size?

    169 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  14 comments  ·  Flag idea as inappropriate…  ·  Admin →
  5. Tool like Azure Storage Explorer but for Data Lake

    Please post a supported or sample tool that provides a GUI and lets you upload and download local files to Azure Data Lake Store. It should be able to authenticate with AAD then display folders and let you upload and download to local files. It should not require any Azure subscription permissions, only WebHDFS permissions.

    I would consider this to be the equivalent of the Azure Storage Explorer... but for Azure Data Lake Store.

    198 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  13 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. Provide a way for the EXTRACT statement to source the records rejected by the "-silent" option

    It would be very useful to allow the EXTRACT statement source rejected records in some way. For instance, when records are rejected by using the "-silent" option in DefaultTextExtractor, it would be useful to shunt those records to a rejected output file. "WITH REJECTED RECORDS AS @mybadrecordrowset" (or similar) as an option on the EXTRACT statement would be pretty awesome to have.

    38 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  7. 17 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    5 comments  ·  Flag idea as inappropriate…  ·  Admin →
  8. Support interactive workloads within ADL Analytics

    Today, U-SQL only supports outputting data into files through batch execution. Enable interactive query capability that delivers results back to the client and w/o landing it into ADL. In addition, tools such as Tableau, PowerBI and other BI tools can connect and issue different queries while interactive ADLA performs necessary computation and delivers results of such computation back to the reporting and BI tools.

    181 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  14 comments  ·  Flag idea as inappropriate…  ·  Admin →
  9. Excel Extractor

    Extractor that pulls from Excel Worksheets in a Workbook!

    48 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  7 comments  ·  Flag idea as inappropriate…  ·  Admin →
  10. Support more wildcards in ADLA file sets

    It would be nice to have more wildcards besides the asterisk in file sets. Suppose we've got two sets of files like eg

    file01.tbl
    file02.tbl

    and

    file0101.tbl
    file0102.tbl
    file0201.tbl
    file0202.tbl

    So it's impossible to select just one of the two sets since the syntax

    @set1 = EXTRACT ..... FROM "/file{*}.tbl" USING .....;

    matches all the files. The proposal is to allow another wildcard like eg ? to mean a single character, so we could do eg

    @set1 = EXTRACT ..... FROM "/file{??}.tbl" USING .....;
    @set2 = EXTRACT ..... FROM "/file{????}.tbl" USING .....;

    Of course the actual syntax/wildcard does not have…

    9 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    3 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. Tool like azcopy but for Data Lake

    The AdlCopy tool only allows you to copy a file from Azure Blob Storage to Azure Data Lake Store currently.

    I would like a tool like azcopy which is able to upload a local file to Azure Data Lake Store and download a file from Azure Data Lake Store to a local file.

    I also need it to accept credentials on the command line (maybe the AAD application client ID and secret?) so that it can be used in automation rather than only being run interactively like AdlCopy.

    31 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  4 comments  ·  Flag idea as inappropriate…  ·  Admin →
  12. Enable code/U-SQL to read header row

    provide functionality to read column row/schema of file dynamically

    47 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  3 comments  ·  Flag idea as inappropriate…  ·  Admin →
  13. Dynamic partition creation

    We currently have to manually create vertical partition buckets. This is fine when we’re creating a fixed number of partition buckets and know what they are upfront. However, when dealing with data with an unknown number of partition buckets, this poses a much bigger challenge and requires some external automation (e.g. a PowerShell script) to add new partitions ahead of inserting data that doesn’t belong to any existing partition bucket.

    It would be great if ADL/U-SQL had an option to dynamically create/add new partition buckets so that we’re not forced to ignore or move data to an ‘unknown’ catchall bucket…

    43 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. Allow dead code

    When I'm debugging, dead code can be valuable so I don't have to comment out huge portions of code when I just want to quickly check something.

    13 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  15. Azure Stack Support

    I would like Azure Data Lake to be available on Azure Stack.

    17 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. Support Event Hubs as stream type data input

    Its easy to store stream data in Azure to skip many times data copy operation in each Azure Services. This is beautiful story to realize lambda-architecture in Azure.

    25 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  4 comments  ·  Flag idea as inappropriate…  ·  Admin →
  17. Azure DataLake Store integration to Azure Stack for cloud+on-prem spanning

    Integrate ADLS to Azure Stack so ADLS may span cloud and on-premise location, in order to be presented as a single endpoint with a single security, metadata and lineage (to come ?) management

    6 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →

    This is an item that will be considered for the longer term. At present we do not have plans on this front. We will continue to solicit customer feedback on this.

  18. Native support for F# with USQL

    It would be great to see some F# support for USQL. I understand that you can already do UDFs in any .NET language (provided you inherit from the required base class or interface etc.) but having F# inline with USQL would be excellent. F#'s lightweight syntax and expression-based syntax would be a natural fit with the SQL section of USQL, and I think would provide a more seamless experience switching when between SQL and .NET code than SQL and C#.

    Pushing this further one could envisage an F# / USQL type provider along the lines of the SQL Client one…

    42 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  19. Support on-premise data storage

    I would like Azure Data Lake to be able to work with data storage that is hosted on-premise.

    7 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  20. Support for Java language

    Add Java support of Azure Data Lake. In our company we have a lot Java developers so it will be useful if they interact with Azure Data Lake efficiently in Java.

    6 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    under review  ·  1 comment  ·  Flag idea as inappropriate…  ·  Admin →
← Previous 1 3 4
  • Don't see your idea?

Feedback and Knowledge Base