Azure Search

Azure Search is a search-as-a-service solution that allows developers to incorporate a sophisticated search experience into web and mobile applications without having to worry about the complexities of full-text search and without having to deploy, maintain or manage any infrastructure

  • Hot ideas
  • Top ideas
  • New ideas
  • My feedback
  1. Approximate Entity Extraction

    Financial institutions are supposed to audit all of their transactions and detect the illegal entities out of free format financial texts by means of a LOB search engine. The list of entities are published by the governmental units such as US OFAC SDN list.

    https://www.treasury.gov/resource-center/sanctions/SDN-List/Pages/default.aspx

    The regular search engines including Azure Search fails to address this need. The blog post below illustrate what is actually expected.

    "Is Google Really Unrivaled When It Comes to Search?"
    https://goo.gl/6Dg1mv

    Below is a demo application to test and highlight the features of such search engine addressing this need. Below is a sample query to…

    7 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Enterprise  ·  Flag idea as inappropriate…  ·  Admin →

    Thanks for the feedback. Azure Search now support the ability to extract entities as part of its new “Cognitive Search” capability. If our built in custom entity skills don’t meet your particular needs, you could also create a custom skill with your own entity extraction code.

    Learn more about Cognitive Search in this Ignite Session:
    https://myignite.techcommunity.microsoft.com/sessions/66240

    Here is a complete list of Cognitive Search Resources:
    https://docs.microsoft.com/en-us/azure/search/cognitive-search-resources-documentation

    Thanks!
    Luis Cabrera
    Azure Search Product Team

  2. Return all the highlights and not just 5

    Today Azure Search returns only the first 5 highlight for each field. It would be great to get all the highlights.

    7 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Query - Search  ·  Flag idea as inappropriate…  ·  Admin →
  3. Allow indexer data source to use custom schema for SQL Azure

    Today we HAVE to use dbo schema for source tables used as data sources by the search indexer. As you know, a lot of enterprises have rules against allowing objects to be created on dbo. This seems like a unnecessary requirement, and surely this could be a simple fix, right? Just let me specify whatever level of qualifier for the container name of my object in SQL Azure...

    6 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Crawlers  ·  Flag idea as inappropriate…  ·  Admin →
  4. Support add-ins for custom file types (e.g. DICOM)

    Would like to have DICOM files indexed for search in the Azure Portal. Support for adding custom logic for parsing arbitrary file types would be sufficient.

    6 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Crawlers  ·  Flag idea as inappropriate…  ·  Admin →

    This functionality is now available to all customers as a part of Cognitive Search feature of Azure Search, using “custom skills”. See:
    - Introduction to Cognitive Search: https://docs.microsoft.com/azure/search/cognitive-search-concept-intro
    - Integrate custom skills: https://docs.microsoft.com/en-us/azure/search/cognitive-search-custom-skill-interface

    Please try it out and give us your feedback.
    Thanks!
    Your Azure Search team

  5. Provide azure specific JSON ignore attribute

    I had a need to ignore properties from the search index (and not defining it as a field threw an error). The provided way to do that is a JsonIgore attribute, but in this instance I also needed to serialize the object for something else where I did need those properties.

    I worked around it by creating another azure search specific class and projecting to that, but it's a bit painful.

    Having an AzureSearchJsonIgnore attribute would mean I could use it without interfering with the rest of my code.

    6 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Client SDK  ·  Flag idea as inappropriate…  ·  Admin →

    The latest version of the .NET SDK (1.0.1-preview) allows for custom serialization via JSON.NET’s JsonConverter and IContractResolver mechanisms. These should be powerful enough to implement your own custom “JsonIgnore” attribute. Examples are available in the SDK tests on GitHub. See the “CanIndexAndRetrieveModelWithExtraProperties” test for details: https://github.com/Azure/azure-sdk-for-net/blob/AutoRest/src/Search/Search.Tests/Tests/IndexingTests.cs

  6. Add ASCII-friendly string decoder to indexer field mapping functions

    Indexing of custom blob metadata to specified schema fields is great, but Azure blob metadata is restricted to only ASCII characters. Currently I can get around this limitation by encoding all blob metadata using URI-encoding (Base64 encoding would also work). However, this means that those fields are not searchable/filterable on their original content.

    If you were to add a new field mapping function that decodes URI (or Base64) encoded values, this problem would go away.

    6 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Crawlers  ·  Flag idea as inappropriate…  ·  Admin →
  7. base64decode method should handle standard strings

    Azure Search indexer base64decode method should handle standard URL Safe Base64 encoded strings, and not the out-of-standard strings returned by classic System.Web.HttpServerUtility.UrlTokenEncode method, which is not even available in latest versions of the framework. This method is returning non-standard strings, as it replaces padding characters '=' with a digit indicating the number of = signs that were removed. Azure Search base64 decode method expects this non-standard strings, rather than standard URL safe Base64 encoded strings, otherwise it breaks with error: "Error applying mapping function 'base64Decode' to field 'aaa': Array cannot be null.\r\nParameter name: bytes". Which means that using JavaScript or…

    6 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Indexing  ·  Flag idea as inappropriate…  ·  Admin →
  8. Show documents on the portal itself

    On azure portal, upon creating index and uploading data we just get to see the count of documents. It would be super helpful to be able to see the data also - like we see in sql table on querying.

    5 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Portal  ·  Flag idea as inappropriate…  ·  Admin →
  9. Ensure data types are consistent with other Microsoft datatypes

    I presume my concern is that almost every Microsoft product seems to have a different set of supported data types and often the same name means different things in different products. In general we've been trying to get people to avoid using "money" in SQL Server and to use "decimal" instead. Similarly I presume that "text" is just a generic name for a bunch of characters but given that it's also the name of a data type that we're trying to move people away from in SQL Server

    5 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    completed  ·  0 comments  ·  Data Types  ·  Flag idea as inappropriate…  ·  Admin →
  10. Support character expansion - e.g., German Eszett - ß

    In .NET text searches, using InvariantCulture, if I search for Straße, I will also get results for Strasse, and vice-versa.

    Azure Search should also support that.

    5 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Indexing  ·  Flag idea as inappropriate…  ·  Admin →
  11. Apply different search terms to different sets of fields

    When I have multiple words to search on, and multiple fields to search, I would like to be able to define which search words should be used to search on which search field.

    To clarify take a look at this request:

    https://[oursearch].search.windows.net/indexes/jobs/docs?$count=true&search=admin london&searchMode=all&searchFields=jobTitle,organisation,description,location&api-version=2015-02-28

    This search terms are admin london. The fields I want to search on are title,organisation,description,location. I know london is a location so I only want to search in the location field for London but should search for "admin" in the other three fields (title,organisation,description).

    4 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Query - Search  ·  Flag idea as inappropriate…  ·  Admin →
  12. 4 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Indexing  ·  Flag idea as inappropriate…  ·  Admin →
  13. Provide searching of phrases

    Without this capability, azure search may not be able to effectively search "dense" content (e.g., msw). Another example is an index I have with scientific journal docs. When I search for "back pain", the best I could do (please correct me if I'm wrong) is:

    text=back,pain&textmode=all

    Even though each query result contains "back" and "pain", most of them are irrelevant and have nothing to do with "back pain".

    Boosting the rank scoring of finding multiple adjacent terms could help but the query results would still mostly be irrelevant.

    4 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Query - Search  ·  Flag idea as inappropriate…  ·  Admin →
  14. Tag Boosting

    The idea is simple: apply a boost to a document if it matches a tag, where matching a tag means that given a field name and a value that’s part of the scoring function definition, we apply the boost if the field value equals the given value or for multi-valued fields if any of the values equals the given value.
    This allows people to tag products as “promotion” and get a generic promo boost or stuff like that.
    If we had conditions on all scoring functions, this could just be constant scoring with a condition. The scenario is so clear…

    4 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Scoring  ·  Flag idea as inappropriate…  ·  Admin →
  15. import data form

    Resolve Import Data Grid Bugginess

    4 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    2 comments  ·  Portal  ·  Flag idea as inappropriate…  ·  Admin →
  16. blob index

    How does the Azure Search Blob Indexer work with billions of blobs? The only feasible way is if it can query the Blob Service to enumerate all the blobs with a LastModified timestamp greater than the last time that the indexer was run. But the Blob Service does not support any such filter (grr why not?!). So how does Azure Search Blob Indexer work? The transaction cost for enumerating billions of blobs would be large.

    Perhaps Azure Search Blob Indexer relies upon that "hack" of using Blob Service logging to detect when blobs are updated. But logging is described in…

    4 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Crawlers  ·  Flag idea as inappropriate…  ·  Admin →

    Hi,

    Please post questions about Azure Search on StackOverflow or our MSDN forum (https://social.msdn.microsoft.com/Forums/en-US/home?forum=azuresearch) to ensure quick response.

    A single blob indexer (or even a single search service) is not going to cope with billions of documents, due, among other reasons, to scalability limits on enumerating blobs as you note below.
    Our advice for scaling blob indexing is to provision multiple datasource/indexer pairs all writing to the same index (with datasources potentially pointing to different storage containers or storage accounts).
    Make sure your search service is scaled to run multiple indexers concurrently; keep in mind that one search unit can run one indexer at a time.

    Hope that helps!
    Your Azure Search team.

  17. Indexing shouldn't fail for all documents when one exceed size limit

    Scenario:
    1. We have a production application using Azure Search with Basic Pricing Tier
    2. The indexer is already configured with these parameters:
    "parameters": {

    "maxFailedItems": 10, 
    
    "maxFailedItemsPerBatch": 5,
    "configuration" : { "failOnUnsupportedContentType": false }

    },
    3. indexing runs once every 60 minutes

    Since we are using the Basic pricing tier, file size limit is 16MB and this is acceptable for the business.

    When a system administrator uploads several documents, and one of them exceeds the 16MB limit, indexing breaks and stops for all documents, even the good ones.

    Logs show this kind of error messages:
    [

    {
    
    "key": null,
    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    2 comments  ·  Bugs  ·  Flag idea as inappropriate…  ·  Admin →

    The desired functionality is already available using ‘indexStorageMetadataOnlyForOversizedDocuments’ configuration property.

    PUT https://[service name].search.windows.net/indexers/[indexer name]?api-version=2016-09-01
    Content-Type: application/json
    api-key: [admin key]

    {
    … other parts of indexer definition
    “parameters” : { “configuration” : { “indexStorageMetadataOnlyForOversizedDocuments” : true } }
    }

    Thank you for using Azure Search!

  18. Create multiple data sources

    Have the option to run multiple data sources will give a extra level of performance and will avoid reinventing the wheel.

    We are working with multiple databases in Azure but we need to combine them and bring a particular result. Multiple data sources will bring the feature which allow us to actually use Search as an really advance search engine. We use DocumentDb and SQL Server in Azure, so it will be just brilliant to have that combination. The posibilities will be endless.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Indexing  ·  Flag idea as inappropriate…  ·  Admin →
  19. Define target language for index

    in terms of internationalisation (internationalization), it would be great if there was a way to indicate a target language in the same way that a document type can currently be indicated when creating a full text index ie: be able to indicate that another column or attribute is holding details of the language. Auto-language detection is great but often isn't enough. What it is good for is multi-lingual documents.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Internationalization  ·  Flag idea as inappropriate…  ·  Admin →
  20. Open source the Azure Search .NET SDK

    It would be helpful if Azure Search team opened source the .NET SDK like Azure Storage team did on GitHub: https://github.com/Azure/azure-storage-net

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Client SDK  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Feedback and Knowledge Base