Approximate Entity Extraction
Financial institutions are supposed to audit all of their transactions and detect the illegal entities out of free format financial texts by means of a LOB search engine. The list of entities are published by the governmental units such as US OFAC SDN list.
The regular search engines including Azure Search fails to address this need. The blog post below illustrate what is actually expected.
"Is Google Really Unrivaled When It Comes to Search?"
Below is a demo application to test and highlight the features of such search engine addressing this need. Below is a sample query to test it.
Below is the slide deck of the presentation in which I explained the need for such a special search engine along with some tips and tricks to develop it.
Below is the very rough draft of the paper I have been currently writing about this search engine.
The LOB which consume the result of this search special engine blocks the transaction having the match result set contains a match with a certain amount of score, for manual investigation. High precision is inherently expected to reduce workforce for investigating false positives. On the other hand, since this is a kind of legal domain, the recall is expected to be near perfect. The matches are also expected to be a fault tolerant.
Not only financial institutions but also flight companies, hotels, police department etc. needs such search engine. It can also be used for product search since the searching product names share similar characteristics with searching person names. There are a few companies out to address this specific need for name matching with very costly enterprise solutions. Yet, I personally found the solution of a couple of them quite weak.
I found no public and open source solution for it and I am planning to finalize the search engine for public use on Azure Marketplace. It may be also called Approximate Entity Extraction Search Engine rather than a simply calling search engine.
I would be very glad to collaborate each other if you consider to develop a search engine like this under Azure Search.
Thanks for the feedback. Azure Search now support the ability to extract entities as part of its new “Cognitive Search” capability. If our built in custom entity skills don’t meet your particular needs, you could also create a custom skill with your own entity extraction code.
Learn more about Cognitive Search in this Ignite Session:
Here is a complete list of Cognitive Search Resources:
Azure Search Product Team