Data Lake

You can use this set to communicate with the Azure Data Lake team. We are eager to hear your ideas, suggestions, or any other feedback that would help us improve the service to bet fit your needs.

If you have technical questions, please visit our forums.
If you are looking for tutorials and documentation, please visit http://aka.ms/AzureDataLake.

  1. Add support for ADLS Gen2 to ADLA

    Would like the ability to use the ADL Analytics job-as-a-service to work with data in ADLS Gen2.

    105 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    14 comments  ·  Flag idea as inappropriate…  ·  Admin →
  2. Assigning ACLs for existing child items

    We need a way to recursively apply ACLs to subdirectories or folders within a Data Lake Storage Gen2 resource, meaning, for them to be inherited for existing items at the time of the ACL creation operation is executed

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. RBAC + ACL Security for ADLS Gen2 is backward

    After reading the docs and speaking to a MS storage rep, I've learned that in ADLS Gen2 RBAC overrides ACL, and you *must* have at least a Reader role assigned via RBAC in order to even see the storage account. This is backwards and broken IMHO - it makes it impossible to allow someone access to only a single folder somewhere in a Filesystem. This is a show stopper for storing any kind of sensitive data (HR, Finance, etc) in ADLS Gen2 where you have a need to totally block access to some data for a person while simultaneously allowing…

    4 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  4. 6 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  5. Support copy of Empty folders in data lake

    Currently while copying multiple folders in data lake gen1/gen2 empty folders are skipped.
    This causes big issue in partitioned data like Parquet/Orc where the corresponding hive tables are corrupted because of missing partitions.
    Apart from this the folders in the datalake are created for a reason even if empty and shouldn't be skipped anyway as it has huge implication on backup-restore approaches.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. C# API

    ADLS Gen2 definitely needs performant and easier to use "native" API's for C# .NET (and other languages), like the BLOB API that initially seemed to work but has now been disabled for Gen2.

    4 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  7. az PS cmdlet for working with ADLS gen1 firewall settings

    We have a need to configure firewall settings for all ADLS gen1 resource in all our subscriptions. This was straightforward for our regular Storage Accounts with the *-AzStorageAccountNetworkRuleSet cmdlets. Could we have similar cmdlets developed for ADLS gen1? This would allow us to report on and reconfigure these resources simply with powershell.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  8. async

    Add async methods to all Microsoft.Azure.DataLake.Store methods. I have to use a sync method to enumerate directory entries, amongst other things..

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  9. Need better information in the Azure Portal and CLI on Data Lake gen2 feature availability

    We were trying to use a gen2 Data Lake as a capture endpoint for an Event Hub.

    The Azure Portal would not let us create a gen2 container as the capture resource for our Event Hub. It gave us an error telling us our selected container name was invalid (see attached) even though we tried a dozen or so valid, random names.
    After some research we found that there is currently no availability for most resources Data Lake gen2 could interact with:

    https://docs.microsoft.com/en-gb/azure/storage/blobs/data-lake-storage-upgrade?toc=%2fazure%2fstorage%2fblobs%2ftoc.json#azure-ecosystem

    There should be better errors shown when trying to use these unavailable connections.

    The error messages seem…

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  10. About Azure Data Lake Analytics, pleaes add function that can access Azure Blob Storage which enabled fire wall.

    I am unsing Azure Blob Storage wich enabled fire wall.
    When I tried to access this Azure Blob Storage with Azure Data Lake Analytics. But It can not access this Azure Blob Storage.
    So, pleaes add function that can access Azure Blob Storage which enabled fire wall.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. Please Make Azure Data Lake Analytics available in US-West

    Our client has a policy to only use services out of the US-West region.
    Are there any plans to make this available in other regions in North America besides US-East and US-Central, particularly US-West?

    If not, could you please put this on your roadmap.

    Thank you.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  12. Add a U-SQL REPL interface to the data lake

    Add a U-SQL REPL interface similar to the Visual Studio C# Interactive Console. Or like the normal script interface you have in SQL Server Management Studio. The current process of running entire scripts is too slow.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  13. 1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  14. 1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  15. Data Lake Analytics Explorer does not have an icon

    See screenshot.
    All the tool window tabs display an icon if not enough place for a label. But Data Lake Analytics Explorer displays nothing

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. 1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  17. Can we get Azure instance metadata for data disks

    Please add metadata for data disk as well so that its very easy to identify OS and Azure end.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  18. Support Event Grid

    Add event grid support to datalake as event grid provider each time that file was added/edited.

    40 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    7 comments  ·  Flag idea as inappropriate…  ·  Admin →
  19. Create Datalake with Folder and Permissions via ARM Template

    The use-case demands to create an EventHub, a Datalake and enable the Capture feature to save incoming data from the Eventhub in the Datalake, directly.

    Currently it is not possible to enable the capture feature on a new created Datalake, because both the folder path and the permissions are not set / created, yet. Setting permissions and creating folders is not possible via ARM templates.

    We want to be able to create a Datalake via an ARM Template and set permissions to it and create the folders.

    12 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  20. U-SQL database project template - Need option to sort the scripts

    Recently, I have started using U-SQL project template and it works good. But I don't have option to sort which files should be merged to final DDL and which one next.

    I assume that whichever is create first it comes first. It would be great if you can provide an option to sort the U-SQL files before it is merged into one DDL file/statements.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    2 comments  ·  Flag idea as inappropriate…  ·  Admin →
← Previous 1 3 4 5 16 17
  • Don't see your idea?

Data Lake

Categories

Feedback and Knowledge Base