Support Resource Partitions with more than 50GB storage
The 50GB storage limitation has been restricting our ability to scale collections since day one.
Almost every other competing tech that I am personally aware of on Azure allows you to pick the "node" storage size.
Cosmos doesn't thus picking a one-size fits all strategy.
I understand that this is a big architectural change for Cosmos (each resource partition has hidden replica sets that need to have matching storage limits etc) but I really believe this should be supported even if 1) I pay more to offset your costs and 2) it's only available with certain restrictions or limitations like for example lower SLA for global distribution or no global distribution etc. We would gladly exchange some of Cosmos DB features for more storage.
This limitation is affecting us in two ways:
1) it's now storage and not compute that is forcing us towards very big numbers of resource partitions which unnecessarily affects our throughput (more weak shards -> more parallelism). If i only need 40K RU to do my job why am I forced to live with 40 resource partitions when 4 would be more than enough?
2) We can't really scale. We would have gladly payed 1mil RU yearly to handle our most demanding tenants (80K-160K users) if it wasn't for the fact that their storage needs are 5-10 times more than what a 1mil RU collection can store. So unfortunately we have been telling them: "Sorry no can do with Cosmos. There's a 16K user limit for you. We can't take your money. Please spend it on a different technology."
Please add support for picking resource partition sizes (50GB, 100GB, 250GB, 500GB) or at the very minimum do some very aggressive compressions.
The size of our physical partitions is driven by the generation of hardware our fleet runs on. This was recently upgraded which allowed us to move to 50GB for physical partitions. We will certainly continue to upgrade our fleet and at some point we will offer increasing physical partition sizes.
What is not on our roadmap is the option to select the size of physical storage. This is something however we will add to our backlog.
I agree. No big company should take Cosmos seriously if it has all kinds of ridiculous limitations. 20 gig per partition? How about 1 tb? 10K RUs per partition? How about 100K? I’m not saying these numbers are necessarily attainable, but the team needs to do better. For the price we’re paying, the limitations as of now are not justified.