How can we improve Azure Search?

Support "swapping" indexes

Let's say you want to make some changes to an index, like changing from tokenized to indexed or from Edm.String to Collection(Edm.String), ... or even re-index all the data with some additional information.

It would be great if we could have a "production" and a "staging" slot in an index (similar to Cloud Services for example) where we can do whatever we need to do in the staging slot, and as soon as we're ready (all data has been indexed), we simply swap the staging slot to the production slot and from that point on the new index is used.

Having it this way would make it easy to manage on our side, especially if Mobile Devices etc... are directly querying to the index without going through our code.

531 votes
Vote
Sign in
(thinking…)
Sign in with: Microsoft
Signed in as (Sign out)
You have left! (?) (thinking…)
Sandrino Di Mattia shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

11 comments

Sign in
(thinking…)
Sign in with: Microsoft
Signed in as (Sign out)
Submitting...
  • Joe T commented  ·   ·  Flag as inappropriate

    This is a great idea and would actually solve a lot of other issues, or at least provide a good workaround.

  • Dean Peters commented  ·   ·  Flag as inappropriate

    I'd like to amplify one of the comments below, in that something along the lines of index aliasing would be exceptionally useful, especially in cases when working with both rolling indexes ... and if y'all ever get around to fully supporting multiple indexes (https://feedback.azure.com/forums/263029-azure-search/suggestions/16700506-search-multiple-indexes-at-once)

  • scott.vickers commented  ·   ·  Flag as inappropriate

    Index management is currently pretty painful with azure search, it is with other services as well, but not quite as bad.

  • Anonymous commented  ·   ·  Flag as inappropriate

    I am also using Sitecore and would highly recommend that this feature becomes supported. For enterprise solutions there is an expectation that no downtime should occur.

  • Anonymous commented  ·   ·  Flag as inappropriate

    We use CMS called Sitecore which works extensively with indexes. It has a concept of swapping for Lucene indexes. Here's how it works:

    You have one index that is a production index. During the swap process, your application creates a new staging index and rebuilds is from a data source. During the rebuild of a staging index, you application still points to the production index. Once the rebuild of a staging index finishes, your application makes sets the staging index as production, and production index as staging, this making the swap.

  • David Bishop commented  ·   ·  Flag as inappropriate

    Guys - just wondering how many more years you'll need to review this?

    Can we expect an answer before the heat-death of the universe?

  • Glenn Gilbert commented  ·   ·  Flag as inappropriate

    We are going to be running a web job periodically to replace an entire dataset on a search index. I'm working to figure out how to replace the entire dataset within Azure Search, without affecting any active traffic.
    I've used Elasticsearch before and it's concept of Index Alias' (https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html) achieves this.

    The idea is that the alias is a URL front door for the search and internally directs to a specified index. When replacing the data set, the builder creates a new index (typically with a datetime stamp in the index name), populates and verifies the index. If that succeeds, the alias is then switched to the new index and all traffic atomically redirects to the new index, leaving the old index to be deleted.

    Does Azure Search have a concept or mechanism that allows the same atomic behaviour or replacing an index?
    Is it as simple as exposing the alias functionality with Azure Search as the facade?

  • Youngjae KIM commented  ·   ·  Flag as inappropriate

    If editting-index feature is not easy to be implemented by design limitation, this could be a workaround.

  • Nick commented  ·   ·  Flag as inappropriate

    Yes, I've been thinking about this too. Alternatively, supporting the ability to rename indexes could be a viable workaround for this. It would mean your production index could be, for example, idx-products, and you could additionally have an idx-products-staging. Then, when you want to swap, you could rename idx-products to something else (eg idx-products-old) and rename idx-products-staging to idx-products. Granted it's not the neatest solution as it can't be done in an atomic operation, but it could work for some.

Feedback and Knowledge Base