How can we improve Microsoft Azure Data Lake?

Support for range partitioning in USQL

Lacking this feature, I have to create distinct partitions for all the dates plus it does not support high precision datetime datatypes. I ran into a similar limitation with HIVEQL on HDINSIGHT clusters.
If you use the same column for partitioning, it overwrites the column data.

I have a 700 million row table which results in 3423 distinct partitions.
Trying to create these partitions @pdate1.......@pdate3423 results in a gigantic script.

The USQL job is stuck on "preparing" phase for 3 hrs and then quits without doing anything.
SEVERITY Info
DESCRIPTION Job was cancelled by Yarn.
MESSAGE Job was cancelled.
result:CompilationFailed

3 votes
Sign in
(thinking…)
Sign in with: Microsoft
Signed in as (Sign out)

We’ll send you updates on this idea

arvind ravish shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

0 comments

Sign in
(thinking…)
Sign in with: Microsoft
Signed in as (Sign out)
Submitting...

Feedback and Knowledge Base