Storage

How can we improve Azure Storage?

You've used all your votes and won't be able to post a new idea, but you can still search and comment on existing ideas.

There are two ways to get more votes:

  • When an admin closes an idea you've voted on, you'll get your votes back from that idea.
  • You can remove your votes from an open idea you support.
  • To see ideas you have already voted on, select the "My feedback" filter and select "My open ideas".
(thinking…)

Enter your idea and we'll search to see if someone has already suggested it.

If a similar idea already exists, you can support and comment on it.

If it doesn't exist, you can post your idea so others can support it.

Enter your idea and we'll search to see if someone has already suggested it.

  • Hot ideas
  • Top ideas
  • New ideas
  • My feedback
  1. Bulk upload to table storage

    To have a fast way of uploading a large amount of data to a storage table. Trying to upload 5M entries (4M distinct partition keys) takes 20 to 60 hours (depending on where I run it) to upload single threaded.
    Batch operations don't help much as they are limited to one partition per batch. Relaxing the restriction on batches to allow multiple partition keys in one batch and making batches larger would both help.
    Alternatively uploading a CSV or JSON file as a single operation. It would be acceptable for this to create a new table.

    6 votes
    Vote
    Sign in
    Check!
    (thinking…)
    Reset
    or sign in with
    • facebook
    • google
      Password icon
      I agree to the terms of service
      Signed in as (Sign out)
      You have left! (?) (thinking…)
      0 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
    • It would be Great if azure table storage can support lua scripting to get the data

      Right now there is no way to execute logic on the columns other than Row key and partition key. It would be great if we can write some logic using lua scrpts which can execute on the server itself and get the results.
      Because it take lot of effort to get the result and filter out at client side.

      1 vote
      Vote
      Sign in
      Check!
      (thinking…)
      Reset
      or sign in with
      • facebook
      • google
        Password icon
        I agree to the terms of service
        Signed in as (Sign out)
        You have left! (?) (thinking…)
        0 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
      • Allow "Prefer: return-no-content" for upserts (i.e. InsertOrMerge/InsertOrReplace)

        As per subject, allow to ignore echo on upsert operations too. There are cases where echo is not important when performing the upsert operation itself even if a row has been modified/replaced (ex: the table row will be read back later by another component).

        3 votes
        Vote
        Sign in
        Check!
        (thinking…)
        Reset
        or sign in with
        • facebook
        • google
          Password icon
          I agree to the terms of service
          Signed in as (Sign out)
          You have left! (?) (thinking…)
          0 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
        • Have a TTL on Table Storage rows, so temporary data gets deleted

          Say you are storing details about logs / orders / page views. You may not care about them after 2 weeks / 1 month / 1 year.

          To save money it would be great to have a job run daily that deletes this data when it is deemed out of data by the system designer.

          AWS has this in DynamoDB where you create a column for an expiration date on tables that will accumulate data that might be temporary by nature. When you create a row you populate this column with the timestamp in the future you want this deleted.…

          33 votes
          Vote
          Sign in
          Check!
          (thinking…)
          Reset
          or sign in with
          • facebook
          • google
            Password icon
            I agree to the terms of service
            Signed in as (Sign out)
            You have left! (?) (thinking…)
            0 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
          • Preallocate table sizes

            In our load we typically create a new table and then fill it with ~100M entries quickly. After that we create some entries, change some, delete some, but the size is growing more slowly.

            The problem is that when we load the ~100M first elements from 0 elements, Azure storage becomes really slow. Very high response times. We think this is because it has to allocate lots of space and because it has to rebalance data between buckets many times in the start. After initial warmup, we can run much higher load against it then in the start.

            What we…

            0 votes
            Vote
            Sign in
            Check!
            (thinking…)
            Reset
            or sign in with
            • facebook
            • google
              Password icon
              I agree to the terms of service
              Signed in as (Sign out)
              You have left! (?) (thinking…)
              0 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
            • Azure table storage should have a way to set the retention policy

              There should be a way to implement a Azure table storage retention policy such that anything older than n number of days can be deleted from table storage.

              32 votes
              Vote
              Sign in
              Check!
              (thinking…)
              Reset
              or sign in with
              • facebook
              • google
                Password icon
                I agree to the terms of service
                Signed in as (Sign out)
                You have left! (?) (thinking…)
                1 comment  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
              • Metrics for the number of partitions

                Today we have a $MetricsCapacityBlob table, and we have metrics on transactions against tables and blobs: https://msdn.microsoft.com/en-us/library/azure/hh343264.aspx

                We would like get metrics for the number of partitions an Azure storage table is scaled over. This would help us scale the number of concurrent parallel reads we are doing against the table.

                4 votes
                Vote
                Sign in
                Check!
                (thinking…)
                Reset
                or sign in with
                • facebook
                • google
                  Password icon
                  I agree to the terms of service
                  Signed in as (Sign out)
                  You have left! (?) (thinking…)
                  0 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
                • Delete a table should be a quicker and non blocking process

                  The Table Delete Process should not leave the recreation of the table disabled.
                  When deleting a big table in table storage, you have to wait an undetermined amount of time to be able to re create that table. The deletion time should be a predictable process, or at least a non blocking process. Maybe during the process, the table can be "renamed" so there is no conflict when trying to re create it.

                  3 votes
                  Vote
                  Sign in
                  Check!
                  (thinking…)
                  Reset
                  or sign in with
                  • facebook
                  • google
                    Password icon
                    I agree to the terms of service
                    Signed in as (Sign out)
                    You have left! (?) (thinking…)
                    0 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
                  • Support geospatial storage in Azure tables

                    Basically the idea is to store data in geospatial format and then allow table queries using geospatial concepts... for example, "points within 5 miles of this location" or "points within this boundary area".

                    SQL Azure supports geospatial storage and DocumentDB does as well. It seems like Azure tables aren't getting the love.

                    11 votes
                    Vote
                    Sign in
                    Check!
                    (thinking…)
                    Reset
                    or sign in with
                    • facebook
                    • google
                      Password icon
                      I agree to the terms of service
                      Signed in as (Sign out)
                      You have left! (?) (thinking…)
                      0 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
                    • Provide backup solution for Table Storage

                      Provide complex solution to backup and restore Table Storage with periodic backup option

                      12 votes
                      Vote
                      Sign in
                      Check!
                      (thinking…)
                      Reset
                      or sign in with
                      • facebook
                      • google
                        Password icon
                        I agree to the terms of service
                        Signed in as (Sign out)
                        You have left! (?) (thinking…)
                        2 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
                      • Add autonumber index with identity increment column like sql server and DocumentDB

                        Don't want to worry about index column like i do on the row key.
                        Please add column id with auto identity increment.

                        64 votes
                        Vote
                        Sign in
                        Check!
                        (thinking…)
                        Reset
                        or sign in with
                        • facebook
                        • google
                          Password icon
                          I agree to the terms of service
                          Signed in as (Sign out)
                          You have left! (?) (thinking…)
                          0 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
                        • Status code returned by InsertOrReplace and InsertOrMerge should reflect if entity was inserted or replaced/merged

                          InsertOrReplace and InsertOrMerge return 204 on success but caller can't tell if entity was inserted or replaced/merged. It would be useful to know so caller can maintain accurate count of entities in the table.

                          15 votes
                          Vote
                          Sign in
                          Check!
                          (thinking…)
                          Reset
                          or sign in with
                          • facebook
                          • google
                            Password icon
                            I agree to the terms of service
                            Signed in as (Sign out)
                            You have left! (?) (thinking…)
                            0 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
                          • Add an InsertIfNotExists

                            There should be an ability to insert a record only if that record does not exist. This would match off the partition & key, and would be the equivalent of checking for the rows existence followed by insert if it did not exist, or do nothing if it did. This is not the same an Upsert or Merge.

                            1 vote
                            Vote
                            Sign in
                            Check!
                            (thinking…)
                            Reset
                            or sign in with
                            • facebook
                            • google
                              Password icon
                              I agree to the terms of service
                              Signed in as (Sign out)
                              You have left! (?) (thinking…)
                              1 comment  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
                            • Provide backup Solution to Take the Backup of Entire Azure Storage Account to Avoid User Errors

                              Provide backup Solution/ option in Azure Management Portal to Take the Backup of Entire Azure Storage Account to Avoid User Errors. This option should have recurring electable schedule.

                              5 votes
                              Vote
                              Sign in
                              Check!
                              (thinking…)
                              Reset
                              or sign in with
                              • facebook
                              • google
                                Password icon
                                I agree to the terms of service
                                Signed in as (Sign out)
                                You have left! (?) (thinking…)
                                0 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
                              • Optimise retrieving of multiple entities across partitions

                                If I request multiple entities, for example with the filter

                                (PartitionKey=1 AND RowKey=2) OR (PartitionKey=3 AND RowKey=4)

                                then a full table scan is performed. Of course, it's more performant to make two queries for fast access to the specific records, but when the client is miles away, perhaps on a mobile device, the extra network traffic for two requests is itself problematic. I'd like the API to internalise the need to split this request into two, retrieve the two entities, then combine them into a single response rather than performing a full table scan. This would provide the best performance…

                                6 votes
                                Vote
                                Sign in
                                Check!
                                (thinking…)
                                Reset
                                or sign in with
                                • facebook
                                • google
                                  Password icon
                                  I agree to the terms of service
                                  Signed in as (Sign out)
                                  You have left! (?) (thinking…)
                                  under review  ·  1 comment  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
                                • Provide "ForceSerializeAttribute" for TableEntity-derived properties and fields

                                  In a sense, it would be the opposite of the existing "IgnorePropertyAttribute" by forcing the serialization of decorated fields and properties (even if it they are private)

                                  38 votes
                                  Vote
                                  Sign in
                                  Check!
                                  (thinking…)
                                  Reset
                                  or sign in with
                                  • facebook
                                  • google
                                    Password icon
                                    I agree to the terms of service
                                    Signed in as (Sign out)
                                    You have left! (?) (thinking…)
                                    0 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
                                  • Please have SAS URLs for a range of tables!

                                    I see people like myself resorting to stupid hacks in order to workaround the fact that there is no way to give access (including create table permission) to a *range* of similarly named tables within a storage account.

                                    I want to do e.g. table.GetSharedAccessSignature(urlPolicy,
                                    tablePrefix: "TablePrefix",
                                    startPartitionKey: _partitionKey,
                                    startRowKey: null,
                                    endPartitionKey: _partitionKey,
                                    endRowKey: null);

                                    Please could you just add a first-class way to do this with SAS tokens?

                                    3 votes
                                    Vote
                                    Sign in
                                    Check!
                                    (thinking…)
                                    Reset
                                    or sign in with
                                    • facebook
                                    • google
                                      Password icon
                                      I agree to the terms of service
                                      Signed in as (Sign out)
                                      You have left! (?) (thinking…)
                                      under review  ·  0 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
                                    • Performance of Azure Table Storage Degrades with More # of Columns in Azure Table

                                      Hi,

                                      When we are having Azure table with 150 + Columns performance of really degrades. For Example : If i have Customer table with 150 Columns and I need to transform Customer Table with specific 25 columns to CustomerDTO object,

                                      Here is the following code used to convert Table object to DTO Object

                                      public IEnumerable<T1> ExecuteQuery<T, T1>( string predicate )
                                      where T : TableEntity, IDomainData, new() //T is Customer Table Entity
                                      where T1 : new() //T1 is CustomerDTO
                                      {
                                      Type typeParameterType = typeof(T);
                                      CloudTable tableReference = tableClient.GetTableReference(typeParameterType.Name);
                                      var query = new TableQuery<T>();
                                      query.SelectColumns = typeof(T1).GetListOfPropertyNames();//Gets the Property list of target…

                                      3 votes
                                      Vote
                                      Sign in
                                      Check!
                                      (thinking…)
                                      Reset
                                      or sign in with
                                      • facebook
                                      • google
                                        Password icon
                                        I agree to the terms of service
                                        Signed in as (Sign out)
                                        You have left! (?) (thinking…)
                                        0 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
                                      • Provide Predicate Overload method for TableQuery Class

                                        Currently in Azure SDK 2.4, to query out Azure table with minimal columns along with predicate condition, we use to do like below

                                        string partitionKey = TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, period);
                                        string rowKey = TableQuery.CombineFilters(TableQuery.GenerateFilterCondition("RowKey", QueryComparisons.GreaterThanOrEqual, startRowKeyCompliance),
                                        TableOperators.And, TableQuery.GenerateFilterCondition("RowKey", QueryComparisons.LessThan, endRowKeyCompliance));
                                        string Condition = TableQuery.CombineFilters(partitionKey, TableOperators.And, rowKey);

                                        var query = new TableQuery<T>();
                                        query.SelectColumns = new list<string>("prop1",....); //the Property list of target Object.
                                        query.FilterString = Condition; //here condition is of string type , can we have option of Predicate type
                                        tableReference.ExecuteQuery(query);

                                        Above Query executes perfect.But ss there a way, where i can pass Expression<Func<T, bool>> predicate has Filter to TableQuery. We Tried by…

                                        6 votes
                                        Vote
                                        Sign in
                                        Check!
                                        (thinking…)
                                        Reset
                                        or sign in with
                                        • facebook
                                        • google
                                          Password icon
                                          I agree to the terms of service
                                          Signed in as (Sign out)
                                          You have left! (?) (thinking…)
                                          under review  ·  1 comment  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
                                        • Add support for ScanIndexForward

                                          Add support for a query parameter like ScanIndexForward (like with Amazon DynamoDB). This parameter would allow us to scan a range from back to front without using a descending row key. This is not the same as an order by, this simply let's me control whether Azure Tables get results in forward or reverse order. The latter is often very useful when you do event sourcing.

                                          7 votes
                                          Vote
                                          Sign in
                                          Check!
                                          (thinking…)
                                          Reset
                                          or sign in with
                                          • facebook
                                          • google
                                            Password icon
                                            I agree to the terms of service
                                            Signed in as (Sign out)
                                            You have left! (?) (thinking…)
                                            under review  ·  2 comments  ·  Tables  ·  Flag idea as inappropriate…  ·  Admin →
                                          ← Previous 1 3
                                          • Don't see your idea?

                                          Feedback and Knowledge Base