We are constantly making improvements to the underlying Answers algorithm. Our latest algorithm release, Andromeda, includes cutting edge improvements that optimize the overall search experience. With this release, our algorithm now has the ability to search semi-structured data. You can now opt-in to access these improvements depending on your configuration.
Introducing Semantic Text Search
A great search engine understands a user's intent and returns the most relevant results. Modern search engines, like Google, Bing, and DuckDuckGo, are really good at this. However, most site search technologies fall short because they only look at the words in the query, not the intent behind it. They use "Keyword Search," which has been around for more than two decades.
Keyword Search has a major flaw: humans use different words to ask the same questions. Consider the following examples:
- What is the population of the United States / how many people live in the USA
- Heart doctor in New York / Cardiologist near NYC
- How many calories in a hamburger / what is the nutritional information for a hamburger
The users behind these searches are looking for the same information, and a keyword-based approach will fail to provide the most relevant results for these searches that are looking for the same information.
Keyword-based systems often employ techniques including TF-IDF (term frequency-inverse document frequency, to determine how important a word is), synonyms, stemming, and lemmatization to improve results, but these hacks are time-consuming and error-prone, and they still do not get to the intent of a user's search.
Yext Answers does things differently. Yext Answers uses a breakthrough technique to search through FAQs in the Knowledge Graph that goes beyond keyword matching. Semantic Text Search analyzes the user's intent in addition to the words in the query. Answers therefore understands what your customers are really looking for, no matter how they phrase it.
FAQs are the most prevalent vertical in Answers, but with just keyword matching it's difficult to search FAQs effectively because keyword matching often cannot understand the intent. Semantic Text Search will help to solve this problem and significantly improve search quality for FAQs.
Semantic Text Search uses BERT - Google's open source machine learning framework for NLP - to represent phrases as points in space, called embeddings. You can visualize this process in 2D through the diagram below:
Instead of looking for overlapping keywords, Semantic Text Search measures the distance between the user's query and the FAQ as a measure of intent. The closer they are, the better the match. Semantic Text Search then calculates the distance to every FAQ in the Knowledge Graph and sorts and ranks search results based on that distance.
Overall, Semantic Text Search will allow customers to use Answers for both structured and semi-structured data, with the added benefit of eliminating the need for synonyms.
Note: Semantic Text Search is available for FAQs only and is currently available in English. Other languages will be added soon.
Answers in Japanese
Following our Summer '20 Release, we launched support for Answers in four new languages: French, German, Italian, and Spanish. With Andromeda, we are extending that support to Japanese as well, to continue to provide official answers globally.
Just as it does in English, Answers now applies advanced natural language processing (NLP) technology to understand questions and return answers in all five of these additional languages. Search experiences powered by Yext Answers in these five new languages can handle searches with complex language-specific attributes, like accents and compound verbs, as well as location-based searches.
If you have any questions or feedback related to the Andromeda Algorithm Update, please stop by the Hitchhikers Community. We have the experts standing by, ready to answer any questions you have.