Ability to delete from Table Storage by Partition
There is no good way to delete multiple entries in Table Storage. All you can do is delete one at a time. For logging tables this can become VERY expensive. It would be great if we had the ability to use Partitions as a way to delete logical groups of data in Table Storage in a single transaction.
This would allow for a partitioning scheme for grouping data in units that can easily be deleted. For example, logging data in WADLogsTable or rolling tables of data captured on a given partition can be archived easily and cleaned up.
MikeWo
shared this idea
We understand this is a top customer ask and as such it is currently on our backlog to be prioritized. We will update when the status changes.
17 comments
-
Brian Burress
commented
The OP is fairly old. As there is some recent activity, I thought it worth chiming in.
I published a package on Nuget (https://www.nuget.org/packages/TheByteStuff.AzureTableUtilities/) which has an option to delete an entire table definition as well as select rows given PartitionKey or TimeStamp information. There is also a command line tool reference using it on GitHub.
This does not offer exactly what you are asking for; however, it may offer an option that helps with what you are trying to do.
-
Aaron
commented
Would really like to be able to set a retention policy on table storage....
-
William Shoap
commented
Can someone advise when or if Azure Data Lake Analytics will support ADL Gen 2? Currently, it only supports Gen1 However, we need ADLA functionality with Gen2. Actually, lots of current customers using the Gen1 platform won't migrate to Gen2 because ADLA support is currently missing. Changing processes over to Databracks, HDInsight, etc may not provide the same functionality or may even be overkill.
-
Karthi
commented
Is it implemented in Azure Cosmos DB Table API ?
-
Scott Helme
commented
Looping back here, still hoping to get an update on this!
-
Rayan Albouni
commented
The post was shared in 2011, MSFT replied in 2016 .. it took 5 years to tell that it is prioritized ..
-
Jim
commented
Even a facade to do this would be better than nothing. Surprised there hasn't been any official comment on this for a while.
-
Scott Helme
commented
Happy 2019! Any update on this one?
-
Scott Helme
commented
Fast approaching 7 years since this was first asked. Any update would be great!
-
Rune Synnevåg
commented
Please answer Microsoft!
-
Scott Helme
commented
2 months on, just looking for an update on this! :)
-
Zhou Chen
commented
This is in fact a top customer request. "Ask" is a verb. Kind regards.
-
Scott Helme
commented
It's been well over a year since the response on this, has there been some progress in that time? The ability to delete by partition key would be awesome, especially for customers like myself who can't structure our data such that partitions are tables instead. Deleting millions of rows with EGTs, after querying them all out, isn't great!
-
Vossekop
commented
This would work even better if we could easily query for a list of partitions in the table.
-
Josh
commented
You can perform a batch delete but that is restricted to 100 rows. I also find it flustering that if I convert a Table Entity to a model for my client and back to a table entity in my data layer, TableBatchOperation Delete never finds my entity. I must instead query for the entity then batch delete the retrieved entity. What is going on internally in TableEntity that is preventing the conversion of entity=>model=>entity? All Delete should care about is the PartitionKey and RowKey. However, this appears to not be the case.
-
Jonathon Rossi
commented
It seems the workaround is still to model your partitions as tables instead, which can be deleted as a single storage operation.
-
zmorris
commented
This or some similar 'mass delete' capability is a must.
Having to retrieve entities first or having to know the entity partition and row key first and then issuing a separate delete request for every entity is not practical (and is expensive) for large historical tables (which is one of the primary use cases for table storage!).