How can we improve Azure Storage?

Auto defragment append blobs

Append blobs are great, however, when a such a blob is generated through a long series of small increments, the read performance of the resulting blob is very poor: from 10x to 20x slower than reading a regular page blob.

The performance problem goes away if the app rewrite the append-blob in large 4MB chunks. However, this process is complicated to setup, and collides with any 'always-on' property of the app.

As append blobs are append-only, it would be much better if Azure was taking care of defragmenting the append blobs on its own; possible through a dedicated API operation that would trigger the operation.

30 votes
Vote
Sign in
Check!
(thinking…)
Reset
or sign in with
  • facebook
  • google
    Password icon
    I agree to the terms of service
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    Joannes VermorelJoannes Vermorel shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

    1 comment

    Sign in
    Check!
    (thinking…)
    Reset
    or sign in with
    • facebook
    • google
      Password icon
      I agree to the terms of service
      Signed in as (Sign out)
      Submitting...
      • Victor NicolletVictor Nicollet commented  ·   ·  Flag as inappropriate

        We benchmarked a full download of two different append blobs. The first was created with a single append of 4MB, the second was created with 16000 appends of 250 bytes each (which corresponds to our typical usage), which is also about 4MB.

        A full download of the first blob, from our fiber connection, takes 750 milliseconds on average. The second blob takes 21 seconds on average, or a 28x increase.

      Feedback and Knowledge Base