Allow me to pay for increased entities/second from a table storage account
Amazon's DynamoDB offers the ability to scale access to a single table, while Azure Table Storage is limited to 5000 entities/account. With Azure Federations now in place, this seems like the last remaining bottleneck for my application as it grows.
Spreading my data across multiple accounts is a suboptimal solution because (1) it requires an additional layer of complexity in my application, (2) these accounts are not free in that I must request more of them from Microsoft myself every time I need more, (3) I'm not guaranteed to get more than 20 (5 to start, 20 if you go through one round with Azure support), and (4) I have to spend time monitoring and modeling when I should start adding new customer accounts to the next table storage account, which involves the guesswork of estimating future loads of customers.
We’ve now announced that all accounts can scale up to 20,000 entities/second (http://msdn.microsoft.com/en-us/library/windowsazure/dn249410.aspx).
Would it be possible to remove all account and table limits and only place a transaction limit of entities/second per PartitionKey?
If data with the same partition key generally resides on the same physical server where physical technical limits exist then wouldn't this be the best place to impose a transactional limit?
I think the 5000 is a tech limits for them right now. They cannot charge you if they cannot deliver
I personally just stumbled across this information and I have to agree 5000 entities per second should be allowed to be scaled in someway.
A way of handling this is to allow us to enable high load transations in the billing. A high load transaction is anything above your 5000 limit and counts seperatly than a normal transaction and is shown on your bill as such. These transactions then can have different allowance / billing details on different packages.
The reason for enabling the feature is some people might not need this but may cause a peak load in 1 second but are happy to make the user wait while it retrys the request.
Alex Bedig commented
Excuse me... 5000 entities/account/second