Support "swapping" indexes
Let's say you want to make some changes to an index, like changing from tokenized to indexed or from Edm.String to Collection(Edm.String), ... or even re-index all the data with some additional information.
It would be great if we could have a "production" and a "staging" slot in an index (similar to Cloud Services for example) where we can do whatever we need to do in the staging slot, and as soon as we're ready (all data has been indexed), we simply swap the staging slot to the production slot and from that point on the new index is used.
Having it this way would make it easy to manage on our side, especially if Mobile Devices etc... are directly querying to the index without going through our code.
Sandrino Di Mattia
shared this idea
12 comments
-
Dan Gøran Lunde
commented
As long as you define your Azure Search setup via code, you can support this scenario today. I am already using an approach where I have two versions of the index. I maintain one "stable" index and all queries are routed to that instance. Then we have a "hot" version where changes to both content and data model/index schema are applied. When indexing is done we route our queries to the latest index instance which is now our new "stable" index. The index roles are thus rotated and updates and changes are now performed on our previous "stable" index.
-
Joe T
commented
This is a great idea and would actually solve a lot of other issues, or at least provide a good workaround.
-
Dean Peters
commented
I'd like to amplify one of the comments below, in that something along the lines of index aliasing would be exceptionally useful, especially in cases when working with both rolling indexes ... and if y'all ever get around to fully supporting multiple indexes (https://feedback.azure.com/forums/263029-azure-search/suggestions/16700506-search-multiple-indexes-at-once)
-
scott.vickers commented
Index management is currently pretty painful with azure search, it is with other services as well, but not quite as bad.
-
Anonymous
commented
I am also using Sitecore and would highly recommend that this feature becomes supported. For enterprise solutions there is an expectation that no downtime should occur.
-
Anonymous
commented
We use CMS called Sitecore which works extensively with indexes. It has a concept of swapping for Lucene indexes. Here's how it works:
You have one index that is a production index. During the swap process, your application creates a new staging index and rebuilds is from a data source. During the rebuild of a staging index, you application still points to the production index. Once the rebuild of a staging index finishes, your application makes sets the staging index as production, and production index as staging, this making the swap.
-
David Bishop
commented
Guys - just wondering how many more years you'll need to review this?
Can we expect an answer before the heat-death of the universe?
-
Glenn Gilbert
commented
We are going to be running a web job periodically to replace an entire dataset on a search index. I'm working to figure out how to replace the entire dataset within Azure Search, without affecting any active traffic.
I've used Elasticsearch before and it's concept of Index Alias' (https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html) achieves this.The idea is that the alias is a URL front door for the search and internally directs to a specified index. When replacing the data set, the builder creates a new index (typically with a datetime stamp in the index name), populates and verifies the index. If that succeeds, the alias is then switched to the new index and all traffic atomically redirects to the new index, leaving the old index to be deleted.
Does Azure Search have a concept or mechanism that allows the same atomic behaviour or replacing an index?
Is it as simple as exposing the alias functionality with Azure Search as the facade? -
Youngjae KIM commented
If editting-index feature is not easy to be implemented by design limitation, this could be a workaround.
-
upvote +3. I came here to write exact same content as this.
-
Aaron West
commented
Any update on this feedback?
-
Nick
commented
Yes, I've been thinking about this too. Alternatively, supporting the ability to rename indexes could be a viable workaround for this. It would mean your production index could be, for example, idx-products, and you could additionally have an idx-products-staging. Then, when you want to swap, you could rename idx-products to something else (eg idx-products-old) and rename idx-products-staging to idx-products. Granted it's not the neatest solution as it can't be done in an atomic operation, but it could work for some.