Storage

How can we improve Azure Storage?

You've used all your votes and won't be able to post a new idea, but you can still search and comment on existing ideas.

There are two ways to get more votes:

  • When an admin closes an idea you've voted on, you'll get your votes back from that idea.
  • You can remove your votes from an open idea you support.
  • To see ideas you have already voted on, select the "My feedback" filter and select "My open ideas".
(thinking…)

Enter your idea and we'll search to see if someone has already suggested it.

If a similar idea already exists, you can support and comment on it.

If it doesn't exist, you can post your idea so others can support it.

Enter your idea and we'll search to see if someone has already suggested it.

  • Hot ideas
  • Top ideas
  • New ideas
  • My feedback
  1. AzCopy should support /MIR option like robocopy

    I would like the performance of AzCopy and the functionality of robocopy /MIR which will incrementally copy a local folder to Azure block blobs including deleting blobs that no longer exist in the local folder. The scenario is wanting to one-way sync a local directory with a blob container. The reverse would also be helpful of one-way synchronizing block blobs to a local directory. Currently I don't believe AzCopy can delete files from the destination.

    263 votes
    Vote
    Sign in
    Check!
    (thinking…)
    Reset
    or sign in with
    • facebook
    • google
      Password icon
      Signed in as (Sign out)
      You have left! (?) (thinking…)
      11 comments  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →

      Thanks for your feedback. We are now planning to add support for /MIR and rsync-like functionality in AzCopy. Unlike them, the sync mechanism will only work on based on time stamp and file size. Deleting files in the destination will be based on an optional flag.

      The first preview of this feature in AzCopy is expected to release around May, 2018.

    • Add a way to copy the contents of an entire storage account to another storage account

      Say your application stores important data in an Azure Storage Account, perhaps spread across both Table and Blob containers.

      It would be very handy to be able to copy the entire contents of a storage account to another storage account. So that you could duplicate your azure storage environment between your stage production environments easily, allowing you to test updates to your application on a copy of your live production data before actually deploying it to production.

      I know that bits and pieces of this functionality exist already, via things like Copy Blobs and Blob Snapshots, but nothing built in…

      142 votes
      Vote
      Sign in
      Check!
      (thinking…)
      Reset
      or sign in with
      • facebook
      • google
        Password icon
        Signed in as (Sign out)
        You have left! (?) (thinking…)
        12 comments  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →

        Thank you for your feedback. We would like to get more feedback about this item before we prioritize. Is the ask here to replicate a storage account on-demand ? Or have a mechanism to always keep two or more storage accounts in sync ?

        Meanwhile, to unblock some of our users we have published a sample that copies blobs from an account to another account: https://docs.microsoft.com/en-us/azure/storage/scripts/storage-common-transfer-between-storage-accounts?toc=%2fpowershell%2fmodule%2ftoc.json

      • AzCopy should support filtering of table entities

        Just like a Pattern parameter for copying blobs, there should be a similar parameter for copying table entities.

        It should be possible to filter both on partition and and row keys.

        I personally find it quite seldom that I need to copy entire table.

        41 votes
        Vote
        Sign in
        Check!
        (thinking…)
        Reset
        or sign in with
        • facebook
        • google
          Password icon
          Signed in as (Sign out)
          You have left! (?) (thinking…)
          2 comments  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →
        • Open Source AzCopy under the same license as the current Storage Client SDK

          Bring all the goodness of open source to AzCopy.

          23 votes
          Vote
          Sign in
          Check!
          (thinking…)
          Reset
          or sign in with
          • facebook
          • google
            Password icon
            Signed in as (Sign out)
            You have left! (?) (thinking…)
            0 comments  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →
          • AzCopy /SetContentType should also tag *.json

            Currently, AzCopy tag *.js and *.css (and a few more) with an empty /SetContentType. It should also tag *.json as application/json.

            11 votes
            Vote
            Sign in
            Check!
            (thinking…)
            Reset
            or sign in with
            • facebook
            • google
              Password icon
              Signed in as (Sign out)
              You have left! (?) (thinking…)
              0 comments  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →
            • Suggest adding a source data timestamp parameter to set as the copy point in time for AzCopy operations.

              Adding a /SourceDataTimeStamp parameter would allow for the ability to only copy data that is at that timestamp or earlier from the source data to the destination. For example, I have many tables and blobs that tie to each other. When I back them all up in a live hosted system, it would be great to say "AzCopy, copy all table and blob data from 2017.09.02 and earlier to another location". This would ensure that if I later restored from that time point, all the data would be be linked up properly.

              10 votes
              Vote
              Sign in
              Check!
              (thinking…)
              Reset
              or sign in with
              • facebook
              • google
                Password icon
                Signed in as (Sign out)
                You have left! (?) (thinking…)
                0 comments  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →
              • Update azcopy packages for ubuntu bionic 18.04

                It might be in progress, but I figured I'd mention in case it's forgotten. Can you guys update the ubuntu bionic packages with azcopy:
                https://packages.microsoft.com/repos/microsoft-ubuntu-bionic-prod/

                10 votes
                Vote
                Sign in
                Check!
                (thinking…)
                Reset
                or sign in with
                • facebook
                • google
                  Password icon
                  Signed in as (Sign out)
                  You have left! (?) (thinking…)
                  0 comments  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →
                • AzCopy include a filter to consider files greater than a specific date

                  AzCopy include a filter to consider files greater than a specific date. This is something which is available in Robocopy/XCopy

                  9 votes
                  Vote
                  Sign in
                  Check!
                  (thinking…)
                  Reset
                  or sign in with
                  • facebook
                  • google
                    Password icon
                    Signed in as (Sign out)
                    You have left! (?) (thinking…)
                    1 comment  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →
                  • Improve Perfromance when /xo is used as it takes longer time when there are more source folders.

                    Add more filters with AzCopy such as Date filters,etc. so that it will help in improving the performance when /xo is used. As currently it needs to scan through all the source files and in case more filters are added the time to use source files would be reduce.

                    8 votes
                    Vote
                    Sign in
                    Check!
                    (thinking…)
                    Reset
                    or sign in with
                    • facebook
                    • google
                      Password icon
                      Signed in as (Sign out)
                      You have left! (?) (thinking…)
                      0 comments  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →
                    • New flag to allow AzCopy to run in backup mode

                      Windows has a relatively unknown backup API that tools like Robocopy can access if you use the /B flag.

                      What it means in reality is that a user with SeBackupPrivilege, running in an admin console with a tool like robocopy can copy data without being restricted by NTFS permissions.
                      For someone that works with File servers every day this would solve a lot of issues.

                      7 votes
                      Vote
                      Sign in
                      Check!
                      (thinking…)
                      Reset
                      or sign in with
                      • facebook
                      • google
                        Password icon
                        Signed in as (Sign out)
                        You have left! (?) (thinking…)
                        0 comments  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →
                      • Add new flag to AzCopy so it would create blobs in lowercase

                        This will be really handy for migration to Azure of a system using NAS and switching it to blob storage - just copy files in lowercase and change system to use lowercase to access to blobs

                        3 votes
                        Vote
                        Sign in
                        Check!
                        (thinking…)
                        Reset
                        or sign in with
                        • facebook
                        • google
                          Password icon
                          Signed in as (Sign out)
                          You have left! (?) (thinking…)
                          0 comments  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →
                        • AzCopy should provide an option to set newly-create blob access policy

                          I use VSTS AzCopy deployment step.
                          I deploy static web files onto my storage account and set container name as current branch name.
                          All is good except there is no access to my blobs :)

                          https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy
                          Here I see only
                          "If the specified destination container does not exist, AzCopy creates it and uploads the file into it."
                          I assume there is no such option now.

                          So I need something like /ContainerDefaultAccessPolicy which is applied only on newly created containers.

                          Thanks

                          3 votes
                          Vote
                          Sign in
                          Check!
                          (thinking…)
                          Reset
                          or sign in with
                          • facebook
                          • google
                            Password icon
                            Signed in as (Sign out)
                            You have left! (?) (thinking…)
                            0 comments  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →
                          • azcopy encryption / decryption needed for upload to BLOB storage

                            AZCopy should allow for data to be encrypted prior to export and decrypted once in BLOB storage. This prevents un-encrypted data from streaming over the wire

                            3 votes
                            Vote
                            Sign in
                            Check!
                            (thinking…)
                            Reset
                            or sign in with
                            • facebook
                            • google
                              Password icon
                              Signed in as (Sign out)
                              You have left! (?) (thinking…)
                              1 comment  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →
                            • AzCopy should by default be part of apps in Powershell container in Cloudshell

                              When you have powershell scripts that uses AzCopy and you want to run them on cloud shell in your cloud drive, it would be great to have AzCopy available there.

                              Especially since the AzureRM.Netcore does not include all the storage commands on Linux and Mac.

                              And since AzCopy is not on mac yet, the cloudshell is needed for more complex management and movements of storage objects.

                              3 votes
                              Vote
                              Sign in
                              Check!
                              (thinking…)
                              Reset
                              or sign in with
                              • facebook
                              • google
                                Password icon
                                Signed in as (Sign out)
                                You have left! (?) (thinking…)
                                1 comment  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →
                              • Allow rate throttling

                                I just tried to use AzCopy to import a storage table into CosmosDB. It immediately faulted with "request rate is too large" even I tried using /NC:1 It would be useful if there was a command line parameter to specify the maximum import rate to avoid this problem.

                                3 votes
                                Vote
                                Sign in
                                Check!
                                (thinking…)
                                Reset
                                or sign in with
                                • facebook
                                • google
                                  Password icon
                                  Signed in as (Sign out)
                                  You have left! (?) (thinking…)
                                  0 comments  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →
                                • ~\Microsoft\Azure\AzCopy

                                  Um, yeah, azcopy should probably not just dump temp journal files into my home directory. XDG is a thing.

                                  1 vote
                                  Vote
                                  Sign in
                                  Check!
                                  (thinking…)
                                  Reset
                                  or sign in with
                                  • facebook
                                  • google
                                    Password icon
                                    Signed in as (Sign out)
                                    You have left! (?) (thinking…)
                                    0 comments  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →
                                  • Azcopy support piping

                                    Here is my scenario

                                    Invoke-sqlcmd -query "Select * from master.sys.table" -ServerInstance myserver -database -mydb | convert-csv | select -skip 2 | .\7z.exe a -si tgzip $gzipedfilename

                                    Now I am taking that $gzipedfilename and uploading to blob storage using azcopy. This is causing huge disk I/O and queue when I am running above script in multiple threads. I am trying pipe Azcopy on the output of above script by doing

                                    | .\7z.exe a -si tgzip -so | Azcopy -----

                                    but I don't know if Azcopy supports piping and how. Can someone help? Or s there any other way to accomplish this…

                                    1 vote
                                    Vote
                                    Sign in
                                    Check!
                                    (thinking…)
                                    Reset
                                    or sign in with
                                    • facebook
                                    • google
                                      Password icon
                                      Signed in as (Sign out)
                                      You have left! (?) (thinking…)
                                      0 comments  ·  AzCopy  ·  Flag idea as inappropriate…  ·  Admin →
                                    • Don't see your idea?

                                    Feedback and Knowledge Base