Add support for ADLS Gen2 to ADLA
Would like the ability to use the ADL Analytics job-as-a-service to work with data in ADLS Gen2.
Russel Loski commented
ADLA provides a different approach to accessing data.
Dave Miyamasu commented
If U-SQL is going away and not supporting Gen 2 storage, then what is MS roadmap for transforming data?
if the sunset is going to happen, what's the idea ... send everyone to aws athena???
I am in no way associated with Microsoft; take what I say as you wish or as an random internet stranger, however we have a significant contract with access to folks there. That being said...
To be clear, it is absolutely not supported per Microsoft employees and forum moderators. I mention this because I saw some older comments mentioning just changing the protocol to abfss. They also WILL be starting to sunset ADLA within 5 years or so per my dedicated ms engineers (I was told this about a year ago). This is really disappointing because I don't want a managed instance of hadoop / databricks. I want quick hitter pay per job easy to learn code and massively scalable.
As such I see no reason this will EVER get implemented as much as we all want it.
Here is a forum link (although older) still holds true: https://social.msdn.microsoft.com/Forums/en-US/5ce97eef-8940-4591-a19c-934f71825e7d/connect-data-lake-analytics-to-adls-gen-2
If your planning on abandoning ADLA (and all your customers that use it) and never supporting ADLS(Gen2) then let us know ASAP.
Adding my 2 cents, the pay-per-job using simple U-SQL syntax [much easier on the brain than full fledged databricks] and thus can be adopted IN-PLACE w/ existing DBA teams that are doing EDW today. That simplified, faster adoption is a huge value. I sincerely hope U-SQL doesn't disappear into obscurity. Certainly ADLA could be adopted into Azure Data Warehouse?
Ardalan Shahgholi commented
Please add the ability to use the ADL Analytics to work with data in ADLS Gen2
Adding data lake storage Gen2 without the analytics seems a backwards step. Is this by design, if so which tool am I expected to use in post processing data to render the results?
I will drop back to gen1 (?) for the mean time.
William Shoap commented
For a superior solution, Microsoft needs to implement ADLA for Gen 2. Lots of customers will be stuck with ADL Gen 1 due to reliance on ADLA and its ease of use. Migrating current processes over to Databrick, HDInsights,etc may by overkill for many customers. Can anyone advise regarding Microsofts intentions to reinclude this functionality or do we just need to keep checking Azure portal to see if it suddenly appears???
It make 100% sense to add support for ADLS Gen2 to ADLA/U-SQL.
ADLA has some significant advantages over data bricks that would make it very useful to have available in ADLS gen 2
*** pay per use and SQL focus brings very low barrier to entry
*** no need for complex additional hadoop/databricks provisionning, scaling & costing decisions
- clear scalability recommendations
- very simple c# extension mecanism
- integrated with common development toolsets (VisualStudio, DevOps)
vikas yadav commented
Please provide clarity on future road map for support of USQL on data lake store gen 2
Mark Freed commented
Another vote for this - we have a classic .Net shop with a ton of C# and Sql. ADLA is a great entry into using big data solutions, so it should be carried forward.
Jeff Moss commented
I'd echo some of the comments on here that I'm working with some clients who want to do U-SQL and C# but not spark/databricks/hadoop...they also prefer to use the Gen2 but clearly this issue causes them a dilemma.
Sergio Sansovini commented
Seems perfect to pe me
Andreas Götz commented
Imo it would make perfect sense to enable ADLA for ADLS gen2. like in some of the other comments mentioned, we dont want to use spark, hadoop or databricks for that matter due to the lack of experience. We do like SQL and C# however, so ADLA would be ideal for us. It is may already and I dont know if its worth waiting for it, use ADLS gen1 or at least ideal use another tool. some feedback would be appreciated.
Joshua Havelka commented
We now have a process built in U-SQL and do not want to have to change it again, so we are stuck in Data Lake Gen1 until ADLA is available in Data Lake Gen2. I would like to move to Gen2 in order to take advantage of the geo redundant backups.
Jeremy Otsap commented
If nothing else I hope you continue to keep USQL alive in the Azure Data Warehouse product line, as well as standard SQL Servers Ie the ability to query blob & ADLS Gen 2 using USQL is a big part of the secret sauce. Many customers do not want the complexity of hadoop or spark
mithun shanbhag commented
We have a client who uses ADLS gen1 (and possibly migrating to gen2 soon) and would prefer to use ADLA for their analytics needs.
However we don't feel comfortable recommending ADLA until the support for ADLS gen2 is available.
William Andrus commented
Just to add more keywords:
Hadoop Filesystem driver