How can we improve Microsoft Azure Data Lake?

Support to Update & Delete operations are in USQL

Support to Update & Delete operations are in USQL; This great feature for Big data environments.

40 votes
Sign in
(thinking…)
Sign in with: Microsoft
Signed in as (Sign out)

We’ll send you updates on this idea

Sakthivel Nachimuthu shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

3 comments

Sign in
(thinking…)
Sign in with: Microsoft
Signed in as (Sign out)
Submitting...
  • Michael Rys commented  ·   ·  Flag as inappropriate

    Thanks for your request. Indeed row-level updates and deletes are useful and it is on our longterm roadmap. Given the current processing architecture, the current work-arounds are:

    1. Use partitioned tables to partition the data so that you can either drop partitions or at least minimize the reprocessing costs.
    2. Reprocess the data by creating a new version with the dropped data deleted.

  • Masaki Itagaki commented  ·   ·  Flag as inappropriate

    In order to comply with various privacy data protection regulations (e.g. US COPPA), we need to delete all rows from one specific user by request. For instance, one user may ask MS to delete all rows for the search service, the speech service, the map service, etc. The request could be achieved only if a row-level deletion is supported. There may be some workarounds including sharding the data by users, but I'm not sure if those workarounds are also scalable in storing and managing the data.

Feedback and Knowledge Base