How can we improve Azure Storage?

Preallocate table sizes

In our load we typically create a new table and then fill it with ~100M entries quickly. After that we create some entries, change some, delete some, but the size is growing more slowly.

The problem is that when we load the ~100M first elements from 0 elements, Azure storage becomes really slow. Very high response times. We think this is because it has to allocate lots of space and because it has to rebalance data between buckets many times in the start. After initial warmup, we can run much higher load against it then in the start.

What we would like is for us to estimate table size when we create a new table (probably also applies for blobs and queues), much like we do when we create a new List<>(numElements) in C#.

When we create table, estimating usage 100M items 10k size each e.g., we would be prepared to pay for that until it is used, but we would also expect good performance during the onboarding the new data.

0 votes
Vote
Sign in
Check!
(thinking…)
Reset
or sign in with
  • facebook
  • google
    Password icon
    I agree to the terms of service
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    Jørgen Austvik shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

    0 comments

    Sign in
    Check!
    (thinking…)
    Reset
    or sign in with
    • facebook
    • google
      Password icon
      I agree to the terms of service
      Signed in as (Sign out)
      Submitting...

      Feedback and Knowledge Base