Ability to delete from Table Storage by Partition
There is no good way to delete multiple entries in Table Storage. All you can do is delete one at a time. For logging tables this can become VERY expensive. It would be great if we had the ability to use Partitions as a way to delete logical groups of data in Table Storage in a single transaction.
This would allow for a partitioning scheme for grouping data in units that can easily be deleted. For example, logging data in WADLogsTable or rolling tables of data captured on a given partition can be archived easily and cleaned up.
We understand this is a top customer ask and as such it is currently on our backlog to be prioritized. We will update when the status changes.
Brian Burress commented
The OP is fairly old. As there is some recent activity, I thought it worth chiming in.
I published a package on Nuget (https://www.nuget.org/packages/TheByteStuff.AzureTableUtilities/) which has an option to delete an entire table definition as well as select rows given PartitionKey or TimeStamp information. There is also a command line tool reference using it on GitHub.
This does not offer exactly what you are asking for; however, it may offer an option that helps with what you are trying to do.
Would really like to be able to set a retention policy on table storage....
William Shoap commented
Can someone advise when or if Azure Data Lake Analytics will support ADL Gen 2? Currently, it only supports Gen1 However, we need ADLA functionality with Gen2. Actually, lots of current customers using the Gen1 platform won't migrate to Gen2 because ADLA support is currently missing. Changing processes over to Databracks, HDInsight, etc may not provide the same functionality or may even be overkill.
Is it implemented in Azure Cosmos DB Table API ?
Looping back here, still hoping to get an update on this!
Rayan Albouni commented
The post was shared in 2011, MSFT replied in 2016 .. it took 5 years to tell that it is prioritized ..
Even a facade to do this would be better than nothing. Surprised there hasn't been any official comment on this for a while.
Happy 2019! Any update on this one?
Fast approaching 7 years since this was first asked. Any update would be great!
Rune Synnevåg commented
Please answer Microsoft!
2 months on, just looking for an update on this! :)
Zhou Chen commented
This is in fact a top customer request. "Ask" is a verb. Kind regards.
It's been well over a year since the response on this, has there been some progress in that time? The ability to delete by partition key would be awesome, especially for customers like myself who can't structure our data such that partitions are tables instead. Deleting millions of rows with EGTs, after querying them all out, isn't great!
This would work even better if we could easily query for a list of partitions in the table.
You can perform a batch delete but that is restricted to 100 rows. I also find it flustering that if I convert a Table Entity to a model for my client and back to a table entity in my data layer, TableBatchOperation Delete never finds my entity. I must instead query for the entity then batch delete the retrieved entity. What is going on internally in TableEntity that is preventing the conversion of entity=>model=>entity? All Delete should care about is the PartitionKey and RowKey. However, this appears to not be the case.
Jonathon Rossi commented
It seems the workaround is still to model your partitions as tables instead, which can be deleted as a single storage operation.
This or some similar 'mass delete' capability is a must.
Having to retrieve entities first or having to know the entity partition and row key first and then issuing a separate delete request for every entity is not practical (and is expensive) for large historical tables (which is one of the primary use cases for table storage!).