A New Way to Search FAQS

Author: Max Davish, Associate Product Manager
Product: Answers
Blog Date: December 2020

FAQs are a hugely important part of Answers. They are the single most viewed, most clicked-on entity across Answers experiences, and they provide a huge benefit to both customers and businesses.

Customers want quick, relevant answers to their questions, without having to click through a complicated website or interact with a chatbot. And businesses want their customers to get this information directly from their website - not by making an expensive call to the support team or, worse, letting a competitor answer on Google.

But searching for FAQs is surprisingly difficult, because there can be so many different ways to ask the same question. Consider these search queries and this FAQ:

A human looking at the FAQ on the right understands intuitively that it's highly relevant to the search queries on the left. But a keyword search algorithm might not return the right result because many of these queries don't actually contain any individual keywords from the FAQ - even though they are still semantically similar.

Sure, you can always solve this by adding synonyms, but this doesn't scale very well because it's impossible to anticipate all the different ways a user could possibly ask a question. Fortunately, there's a better way.

Introducing Semantic Search

Semantic search is a new way of searching for FAQs. Instead of looking at keywords, it measures the similarity in meaning between two questions. Answers does this using BERT - the same revolutionary natural language processing technology that powers other Answers features like location detection and entity recognition.

It works by using BERT to transform questions into points in high-dimensional space, called embeddings. In this space, two questions that mean the same thing will be close together, even if they don't contain any of the same words. So instead of looking at overlapping keywords, we instead search by looking for the FAQs that are the "closest" to the user's question.

What does this mean for Administrators? It means that FAQ search will no longer require managing lots of keywords and synonyms. For example, you won't need to add a synonym between "spread" and "transmitted"; BERT innately understands that these words mean the same thing.

This helps make life better for both users and Administrators. Users can ask complex questions in different ways and still get relevant results, and Administrators won't need to do as much manual configuration of their experience. It will just work!

Semantic Search in Action

Let's see a few examples of semantic search in action. Here's one from Cox Residential, a US-based telecommunications company:

In this query, BERT understands that "virus protection" is related to "security", so instead of showing a list of links, we show the FAQ the user was looking for. No synonyms necessary!

Even when there are keyword matches, semantic search can help understand which keyword matches are more important. For example:

In this example, there are two FAQs that both contain the word "support", and keyword search has no way of knowing which is more relevant to the user's question. But BERT understands that the user isn't asking about modems. They want to talk to someone, so it brings the second FAQ to the top position.

Here's another example from Cherry Creek Mortgage, a US-based mortgage lender:

In this example, BERT understands that when the user asks about the "loan portal", they're trying to access their personal information online, so it returns the exact right FAQ instead of a handful of irrelevant ones that simply contain the word "loan".

Here's another example from Bucknell University:

Normally, to get this query to work you might add a synonym between "doctor" and "pre-med". But with semantic search, there's no need. BERT understands innately that "doctor" and "pre-med" are related concepts.

Here's one last example from Bliss World, a multi-channel spa and retail product company.

Once again, instead of forcing the user to sift through irrelevant results that matched on the product's name - Clear Genius - semantic search identifies the exact right FAQ to answer the user's question, even though it has the same amount of keyword matches as other FAQs.

Using Semantic Search

To use semantic search on your Answers experience, simply navigate to the new Answers Configuration UI and activate Semantic Text Search on the FAQ Name field:

Alternatively, you can set "semanticTextSearch": true under this field's configuration within the JSON editor!

Frequently Asked Questions

(Very meta, we know...)

Does this only work for FAQs?

Yes, right now Semantic Search only works for FAQs. The reason is that FAQ names and user queries are naturally similar, so we can use the same BERT model to understand them both. This isn't necessarily true for other fields, so this type of semantic search doesn't work as well. But in the future we plan on adapting this technology to other entities and fields too!

Can I still use textSearch on other fields for my FAQs?

Yes! You can still search other fields like the keywords field or any custom fields you have. Semantic Search works in conjunction with either text search or NLP filters on other fields. When text search is activated on other fields, the algorithm mixes the signals from both fields to determine which FAQs are most relevant.

Does semantic search work with synonyms?

Yes! Semantic Search doesn't require synonyms, but it still considers synonyms when searching for FAQs. More specifically, Semantic Search will never require synonyms for words that are naturally similar, like "fast" and "quick".
However, you might still want to use synonyms for words that are not naturally similar, or that are only similar in the context of your particular Answers experience.

What if I am using a custom entity type for FAQs? How do I use Semantic Text Search?

Semantic Search only works on the built-in FAQ entity, so you'll need to move data from your custom FAQ entity to the built-in one. This can be accomplished easily by exporting and re-uploading your data.

Semantic Text Search seems to be returning a lot more FAQs than previously. Is this by design?

Yes — because semantic search doesn't require any individual keyword matches, it tends to return more results than textSearch, though not always. You can set a maximum number of results to show on the front-end, using the limit parameter on the results configuration.

All Blog Posts

Determine Location Intent in a Search Engine

Max Shaw, VP Product

One of the most common use cases for a search engine is finding something by a "location". Here are some basic examples: Cardiologist near Green Bay, Notary near me, Restaurants open now. These are all pretty simple queries, but getting these to work in a search engine is much more complex than you might imagine.

4 Methods for Increasing Site Search Clicks

Rick Swette, UX Research

We know good search drives business impact. It increases conversions and transactions, reduces search bounce rate, and boosts overall customer satisfaction. So, how do we get more people to trust and use site search? We embarked on a study to find this out.

How to Measure the Success of Your Site Search

Basil Polsonetti, Data Insights

Most brands know that site search is a feature their website should have, but unless the site is dominated by e-commerce, it’s often relegated as a check-the-box task when building a new website.

The Danger in Document-Level Sentiment Analysis

Calvin Casalino, Senior Product Manager

In order for your feedback to become an actionable item to help businesses provide a better experience, they need a way to analyze the granular content of all of their reviews, at scale.

Deep Dive into Duplicate Suppression

Dee Luo, Product Manager

Brands know the importance of having accurate information across all the apps, maps, and directories where consumers are searching for information. In a perfect world, powering that brand data and managing each of these listings would be enough to ensure that consumers consistently get the answers they're searching for.

Yext Answers Algorithm Update: Milky Way

Max Shaw, VP Product

Yext Answers is constantly improving it’s search algorithm to provide more relevant results over time. Milky Way is the first official upgrade to the Answers Algorithm and includes a series of important upgrades to provide better search precision and recall.

GMB API Update - Dedicated Food Menus

Dee Luo, Product Manager

On August 24, 2020, Google launched version 4.7 of its Google My Business (GMB) API. This update includes enhancements to how your restaurant locations can sync and display food menus on Google.

Structuring Your Knowledge Graph

Jessie Yorke, Yext Administrator

In this post we are going to discuss strategy and give you some tools to effectively think about structuring your own brand's Knowledge Graph!

Welcome to the Hitchhikers Program

Liz Frailey, VP Developer & Admin Experience

Welcome to Hitchhikers! We are so excited to have you join our mission of creating amazing search experiences for brands of all sizes.

Introducing: Yext Answers Plugin for WordPress

Alex Barbet, Product

Businesses of all sizes use both WordPress and Yext to build amazing client experiences, and as more and more brands around the world add the Yext Answers bar to their WordPress powered sites, we wanted to provide a way to drive their time-to-value even faster.

Yext’s Fall ‘20 Release is Now Live!

Nick Oropall, Senior Product Marketing Manager

For those of you who are new to Hitchhikers — Welcome to Yext's new training platform & community! Hitchhikers will be the home for all of Yext's product and release updates moving forward so we encourage you to create a free user and check out the platform!

Meet the Hitchhikers Team: Alyssa Hubbard

Alyssa Hubbard

Alyssa Hubbard began at Yext in the Upward Rotational Program. Now she is full-time on the Hitchhikers team, working to build a platform to empower our community of Yext power users.

Yext Answers Algorithm Update: Andromeda

Allie Allegra, Senior Product Marketing Manager

We are constantly making improvements to the underlying Answers algorithm. Our latest algorithm release, Andromeda, includes cutting edge improvements that optimize the overall search experience. With this release, our algorithm now has the ability to search semi-structured data. You can now opt-in to access these improvements depending on your configuration.

WCAG and Search: Developing an Accessible Search Experience

Rose Grant, Associate Product Manager

What’s WCAG? WCAG stands for the Web Content Accessibility Guidelines (WCAG). WCAG is not always black and white; its rules often have a variety of interpretations.

Meet the Hitchhikers Team: Amani Farooque

Amani Farooque, Program Development Manager

Wonder what's behind the avatar for some of those people you see constantly on the Hitchhikers Community? My name is Amani Farooque and I work as a Program Development Manager at Yext, managing the Hitchhiker Platform and focusing on enhancing developer and administrator experience. Read more to learn more about me and get a behind-the-scenes glimpse of what maintaining the program entails!

Now Available: Shopify Product Catalog Sync for Yext

Lilly Fast, Senior Business Development Manager

Shoppers have questions about your products, and your ability to answer will determine if they buy or if they bounce. But with rapid changes to your business, it can be hard to keep your product information consistently up-to-date.

Now Available: Yext Product Catalog Sync for Magento Commerce, an Adobe Company

Lilly Fast, Senior Business Development Manager

Shoppers have questions about your products, and your ability to answer will determine if they buy or if they bounce. But with rapid changes to your business, it can be hard to keep your product information consistently up-to-date.

A New Way to Search FAQS

Max Davish, Associate Product Manager

Semantic search is a new way of searching for FAQs. Instead of looking at keywords, it measures the similarity in meaning between two questions. Answers does this using BERT - the same revolutionary natural language processing technology that powers other Answers features like location detection and entity recognition.

How to Build Discoverable and High-Converting Landing Pages

Sonia Elavia, Senior Product Marketing Manager

Search is a massively important marketing channel. Fifty-three percent of website traffic comes from organic search, so it’s critical that businesses optimize for these experiences. But how do you actually build out a strong presence in organic search? Having search-optimized landing pages is critical.

Exporters: From Yext to Your Listings

Calvin Casalino, Senior Product Manager

Our Listings delivery pipeline ensures your data stored in the Knowledge Graph appears on Listings everywhere consumers are asking questions. How do we make sure your data is updated on all publishers as quickly as possible while still ensuring data is formatted properly for each endpoint? Yext’s Listings exporters.

2020: Hitchhikers Year in Review

Liz Frailey, VP Developer & Admin Experience

2020 has been a rollercoaster of a year for everyone for a multitude of reasons. On the Hitchhikers Team, we were able to overcome some of the year's obstacles to really transform the program.