How to Measure the Success of Your Site Search

Author: Basil Polsonetti, Data Insights
Product: Answers
Blog Date: June 2020

Background

Most brands know that site search is a feature their website should have, but unless the site is dominated by e-commerce, it's often relegated as a check-the-box task when building a new website. This is evidenced by the many brands who have buried their magnifying glass in the top-right corner of their home page, almost begging for it to not be clicked — or even worse, only accessible via a specific link in the footer. Once there, the customer is led into a search experience far inferior to the modern capabilities of Google and voice search they have become accustomed to, relying on string-matching against landing pages buried deep within the sitemap. While this approach may have been sufficient a decade or two ago, when the website effectively served as a brand's online brochure, today's consumers seek instantaneous answers and will turn elsewhere when their questions go unanswered, whether that's Google, Alexa, Siri, or a competitor's website.

Challenge

Since we started building Yext Answers, we realized that one of the biggest challenges in site search is defining and measuring the success of a first-party search experience. Unlike organic search, where KPIs, such as "drive more site traffic", are clear, universally understood, and longstanding, site search has long been overlooked as a strategic asset to achieve business goals outside of the context of pure e-commerce. To add to this, the vast majority of existing site search technologies are keyword-based, so even the more sophisticated analytics engines will only provide surface-level customer trends. Brands are unable to get a clear picture of both the granularity of consumer searches, as well as how consumers interact with specific search results. NLU in Yext Answers understands the nuances of customer intent at scale. Our belief is that modern-day site search should include a robust source of analytics that allows you to answer the following questions:

  • What questions do customers have for my brand?
  • How can I improve my search results in real-time?
  • How many conversions has my site search driven?
  • How many questions did search deflect from other (and often more costly) support channels?

Web Analytics Platforms to the Rescue….Unless?

Web analytics platforms such as Google Analytics (GA) are great tools for understanding where your web traffic came from and what your visitors did once they got to your site. For that reason, they have been widely adopted by Web Analytics Managers across the globe. According to BuiltWith, 69.5% of the 10,000 top-trafficked sites globally use GA, while 54.6% of the top million sites are leveraging the platform. GA enables you to configure site search, which pulls in analytics around top searches and other stock metrics such as the following:

  • Sessions with Search = The number of sessions that used your site's search function at least once.
  • Percentage of sessions that used internal search = Sessions with Search / Total Sessions.
  • Total Unique Searches = The total number of times your site search was used. This excludes multiple searches on the same keyword during the same session.
  • Results Pageviews / Search = Pageviews of search result pages / Total Unique Searches.
  • Search Exits = The number of searches made immediately before leaving the site.
  • Percentage of Search Exits = Search Exits / Total Unique Searches
  • Search Refinements = The number of times a user searched again immediately after performing a search.
  • Percentage Search Refinements = The percentage of searches that resulted in a search refinement. Calculated as Search Refinements / Pageviews of search result pages.
  • Time after Search = The amount of time users spend on your site after performing a search. This is calculated as Sum of all search_duration across all searches / (search_transitions + 1)
  • Search Depth = The number of pages viewed after performing a search. This is calculated as Sum of all search_depth across all searches / (search_transitions + 1)

While one can track these metrics individually over time to measure the impact of updates to their search algorithm, they are not great at helping us understand the value of a site search experience or contextualizing success for your customers. They may have been helpful for the site search of old, but the modern search journey is a bit more complex than it was back when GA first rolled this out.

Let's take a look at why these GA metrics may not sufficiently measure success in the examples below.

Search Exits — Good or Bad?

In this example, the customer has a very clear question about the benefits of their Larimar Reserve credit card. The answer to the question is also very straightforward: $300 annual travel credit, 3x points on travel and dining, and 50% more in travel redemption. Assuming the customer had no other questions, they would likely close their browser tab/window or navigate on to a new site for a new task.

The question is — should this be viewed as a successful search? Using traditional GA site search metrics referenced above, this would be classified as a Search Exit and viewed as an indicator of a poor search experience. Someone searched and then immediately left your site — that has to be a bad thing, right?

At Yext, we posit that this is, in fact, a resounding success! A customer came to your site with a question. You answered that question with no human intervention necessary, avoiding any potential support costs, and the customer moved on with their day. All this contributes toward a positive customer experience, as 41% of users say the most important aspect of good customer service is getting issues resolved quickly, and 73% of customers prefer to use a company's website instead of social media, SMS, and live chat for support. Check and check.

Search Refinements — Reformulation or New Search?

Piggybacking on the example above, let's say that the same customer wanted to change gears entirely and ask a separate question about Silver's branch locations:

For the query "Branches in New York, NY", the intent of this search is probably to locate the nearest Silver branch and then call or get directions. With traditional site search metrics, this would be counted as what is referred to as a "search refinement." Without context, a search refinement sounds like a bad thing. In the case of legacy site search, a searcher presumably had to reformulate their search because their first attempt did not return results that satisfied their question. However, in this particular case, the search refinement actually validates that the customer had such a positive experience with their first search, and subsequently decided they would ask another question.

This is not all to say that traditional search metrics should be thrown out the window, but rather, they should be viewed in the context of the question being asked. That is why we are making it incredibly easy with Yext Analytics to understand user engagement dimensioned at the session and search query level.

The Value of Great Site Search

No matter the customer scenario, great site search should engage and fulfill a users' intent as effectively as possible. We built Yext Answers to provide brand websites with an exceptional search experience to meet ever-increasing consumer expectations. The value of the product can be broken down into three buckets:

  1. Increase Conversions
  2. Reduce Customer Support Costs
  3. Leverage Customer Intelligence

Increase Conversions

Yext Answers is unique in that it returns structured answers to customer questions, just like Google. Along with the rich results comes rich analytics. Measuring the volume and types of clicks you receive on your search results has never been easier; simple dashboards help you understand the number of clicks you get, along with the breakdown of click type:

  • Click to Call
  • Get Directions
  • Call-to-Action Click
  • Buy Now
  • Apply Now
  • RSVP
  • More! Leveraging Yext Conversion Tracking, you can set up tag-based tracking to attribute website conversions to your Answers experience or you can assign values to each of these click types to directly calculate the total conversion value of all the clicks you accumulate in the platform.

Reduce Customer Support Costs

Customers have a lot of questions, but the majority of these questions are quite similar. This high volume of simply-answered questions should be accessible for consumers to discover on your website. When they can't answer these on their own, they will typically turn to more costly channels, such as a call center or live chat. While they ultimately get their questions answered, these also incur an incremental cost for each interaction. In fact, Oracle and Forrester estimate that each live chat costs $5, while a call can cost as much as $12.

Leveraging the FAQ entity type in Yext's Knowledge Graph and Answers allows your brand to answer these common questions at scale, saving money with each FAQ that is read by a customer. Yext Analytics provides simple reporting on the number of FAQs your customers have read, which makes estimating support cost savings a simple calculation.

Leverage Customer Intelligence

One of the more unique features of Yext Answers is the ability to see live queries in real-time as consumers are searching. When customers have questions, Yext Answers listens, understanding their intent and recording in real time. Identifying new trends and patterns in search behavior allows your Yext Administrator to then add new entities to your Knowledge Graph so that your site can answer those questions. This feedback loop encourages you to improve your search quality, and you can then leverage those same Knowledge Graph entities for use across additional endpoints in the search ecosystem, whether that is a unique landing page, a location listing, or a job posting. Uncovering this customer intent on your first-party platform saves your brand money on potential survey/intent vendors, but more importantly, you can attribute the incremental value of these new endpoints back to the source of that intelligence — Yext Answers.

All Blog Posts

Determine Location Intent in a Search Engine

Max Shaw, VP Product

One of the most common use cases for a search engine is finding something by a "location". Here are some basic examples: Cardiologist near Green Bay, Notary near me, Restaurants open now. These are all pretty simple queries, but getting these to work in a search engine is much more complex than you might imagine.

4 Methods for Increasing Site Search Clicks

Rick Swette, UX Research

We know good search drives business impact. It increases conversions and transactions, reduces search bounce rate, and boosts overall customer satisfaction. So, how do we get more people to trust and use site search? We embarked on a study to find this out.

How to Measure the Success of Your Site Search

Basil Polsonetti, Data Insights

Most brands know that site search is a feature their website should have, but unless the site is dominated by e-commerce, it’s often relegated as a check-the-box task when building a new website.

The Danger in Document-Level Sentiment Analysis

Calvin Casalino, Senior Product Manager

In order for your feedback to become an actionable item to help businesses provide a better experience, they need a way to analyze the granular content of all of their reviews, at scale.

Deep Dive into Duplicate Suppression

Dee Luo, Product Manager

Brands know the importance of having accurate information across all the apps, maps, and directories where consumers are searching for information. In a perfect world, powering that brand data and managing each of these listings would be enough to ensure that consumers consistently get the answers they're searching for.

Yext Answers Algorithm Update: Milky Way

Max Shaw, VP Product

Yext Answers is constantly improving it’s search algorithm to provide more relevant results over time. Milky Way is the first official upgrade to the Answers Algorithm and includes a series of important upgrades to provide better search precision and recall.

GMB API Update - Dedicated Food Menus

Dee Luo, Product Manager

On August 24, 2020, Google launched version 4.7 of its Google My Business (GMB) API. This update includes enhancements to how your restaurant locations can sync and display food menus on Google.

Structuring Your Knowledge Graph

Jessie Yorke, Yext Administrator

In this post we are going to discuss strategy and give you some tools to effectively think about structuring your own brand's Knowledge Graph!

Welcome to the Hitchhikers Program

Liz Frailey, VP Developer & Admin Experience

Welcome to Hitchhikers! We are so excited to have you join our mission of creating amazing search experiences for brands of all sizes.

Introducing: Yext Answers Plugin for WordPress

Alex Barbet, Product

Businesses of all sizes use both WordPress and Yext to build amazing client experiences, and as more and more brands around the world add the Yext Answers bar to their WordPress powered sites, we wanted to provide a way to drive their time-to-value even faster.

Yext’s Fall ‘20 Release is Now Live!

Nick Oropall, Senior Product Marketing Manager

For those of you who are new to Hitchhikers — Welcome to Yext's new training platform & community! Hitchhikers will be the home for all of Yext's product and release updates moving forward so we encourage you to create a free user and check out the platform!

Meet the Hitchhikers Team: Alyssa Hubbard

Alyssa Hubbard

Alyssa Hubbard began at Yext in the Upward Rotational Program. Now she is full-time on the Hitchhikers team, working to build a platform to empower our community of Yext power users.