How can we improve Azure Storage?

← Storage

Ability to delete from Table Storage by Partition

There is no good way to delete multiple entries in Table Storage. All you can do is delete one at a time. For logging tables this can become VERY expensive. It would be great if we had the ability to use Partitions as a way to delete logical groups of data in Table Storage in a single transaction.

This would allow for a partitioning scheme for grouping data in units that can easily be deleted. For example, logging data in WADLogsTable or rolling tables of data captured on a given partition can be archived easily and cleaned up.

779 votes
Vote
Sign in
(thinking…)
Sign in with: Microsoft
Forgot password?
Create a password
Signed in as (Sign out)
Close
Close
You have left! (?) (thinking…)
MikeWo shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

20 comments

Sign in
(thinking…)
Sign in with: Microsoft
Forgot password?
Create a password
Signed in as (Sign out)
Close
Close
Submitting...
An error occurred while saving the comment
  • Scott Helme commented  ·   ·  Flag as inappropriate

    Just circling back on this one and hoping for an update now as it has been over 9 years since it was opened and this is a 'Top Idea' for Azure Storage.

  • Brian Burress commented  ·   ·  Flag as inappropriate

    The OP is fairly old. As there is some recent activity, I thought it worth chiming in.

    I published a package on Nuget (https://www.nuget.org/packages/TheByteStuff.AzureTableUtilities/) which has an option to delete an entire table definition as well as select rows given PartitionKey or TimeStamp information. There is also a command line tool reference using it on GitHub.

    This does not offer exactly what you are asking for; however, it may offer an option that helps with what you are trying to do.

  • Aaron commented  ·   ·  Flag as inappropriate

    Would really like to be able to set a retention policy on table storage....

  • William Shoap commented  ·   ·  Flag as inappropriate

    Can someone advise when or if Azure Data Lake Analytics will support ADL Gen 2? Currently, it only supports Gen1 However, we need ADLA functionality with Gen2. Actually, lots of current customers using the Gen1 platform won't migrate to Gen2 because ADLA support is currently missing. Changing processes over to Databracks, HDInsight, etc may not provide the same functionality or may even be overkill.

  • Rayan Albouni commented  ·   ·  Flag as inappropriate

    The post was shared in 2011, MSFT replied in 2016 .. it took 5 years to tell that it is prioritized ..

  • Jim commented  ·   ·  Flag as inappropriate

    Even a facade to do this would be better than nothing. Surprised there hasn't been any official comment on this for a while.

  • Scott Helme commented  ·   ·  Flag as inappropriate

    It's been well over a year since the response on this, has there been some progress in that time? The ability to delete by partition key would be awesome, especially for customers like myself who can't structure our data such that partitions are tables instead. Deleting millions of rows with EGTs, after querying them all out, isn't great!

  • Vossekop commented  ·   ·  Flag as inappropriate

    This would work even better if we could easily query for a list of partitions in the table.

  • Josh commented  ·   ·  Flag as inappropriate

    You can perform a batch delete but that is restricted to 100 rows. I also find it flustering that if I convert a Table Entity to a model for my client and back to a table entity in my data layer, TableBatchOperation Delete never finds my entity. I must instead query for the entity then batch delete the retrieved entity. What is going on internally in TableEntity that is preventing the conversion of entity=>model=>entity? All Delete should care about is the PartitionKey and RowKey. However, this appears to not be the case.

  • zmorris commented  ·   ·  Flag as inappropriate

    This or some similar 'mass delete' capability is a must.

    Having to retrieve entities first or having to know the entity partition and row key first and then issuing a separate delete request for every entity is not practical (and is expensive) for large historical tables (which is one of the primary use cases for table storage!).

Storage: Tables

Categories

Feedback and Knowledge Base

(thinking…)
Hosted by UserVoice