Shop search analytics: what to log, and the zero-result fixes to ship first

Most shops log that a search happened. Far fewer log whether it worked. The measurement that does is a loop: every search writes one event with the query, its filters, the result count and a join key; every click writes a follow-up event that points back to that key; and once a week someone reads the queries that returned nothing or got no click and turns them into a fix. This article describes that loop: the event schema, five metrics with their definitions and first actions, a correction for position bias, a way to group failures, and the limits on what you should collect.

The loop does not replace retrieval work. Matching, query rewriting and a clean catalogue decide what a search can find, and semantic product search for B2B covers that side. Analytics shows which of those problems shoppers actually hit. The metric definitions below follow how Algolia documents its analytics product, and the event shape follows the open User Behavior Insights (UBI) specification, so the design does not depend on one vendor.

What should you fix first?

Fix the logging gaps first, then the zero-result queries, then the searches without clicks, and only then the ranking. Every other number is only as good as the events behind it, so logging comes first. Zero-result queries come next because they are the cheapest failures to find, and many are vocabulary gaps with a known fix. In the first two weeks of a retro-fit the goal is not a dashboard. It is one complete week of events that you can export, read and act on.

A workable order for a shop without search analytics runs as follows. Add a search event and a click event that share a query_id. Export one week of zero-result queries and read them by hand. Ship three to five synonym or redirect fixes. Add the no-click view. Start an experiment only after the baseline has been stable for a few weeks. Each step is useful on its own, so the full instrumentation does not have to exist before you learn something.

When does search analytics mislead?

Search analytics misleads when traffic is too thin to show a difference, when a single rate is read as a verdict, when seasonality is ignored, and when a few repeat trade buyers are averaged in with everyone else. Each of these is common in B2B shops, where a small group of customers often runs most searches, using part numbers and saved carts.

• Too little traffic. A change that moves a rate by a few points needs enough searches per variant before the result means anything. For a category with a few hundred searches a week, count weeks rather than days, and accept that some questions will stay open.
• CTR as a verdict. A click shows interest, not success. A shopper who clicks the first result, sees the wrong part and leaves has produced a click and a failed search.
• Seasonality. Zero-result rates for seasonal goods or spare parts move with the calendar, so compare the same weeks year on year, or like with like, rather than the last week against a quiet one.
• B2B traffic skew. Repeat buyers who search by part number can dominate the averages. Where your data allows it, segment by account type before reading a rate as a shop-wide result.
• Logs you should not keep. Some fields do not help the loop and add legal risk. The privacy section explains which ones to leave out.

What should one search event contain?

One search event records the query and the context it ran in. Separate events record what the shopper did next, and the two are joined by a query_id that the search service returns with each response. The UBI specification names the fields that make a search log joinable: query_id, the query text, filters, page, a client_id that stays stable when a visitor moves from anonymous to logged in, the position on click events, and the object identifier, such as a SKU. The UBI specification uses an item click on a SKU as its example. A sketch of the two events for a shop looks like this:

// search event
{
  "query_id": "3f9c0b7e-6d2a-4c1e-9a55-2b8f1d7e4a10",
  "client_id": "c-7b1e44",
  "user_query": { "query": "m8x25 hex bolt a2" },
  "filter": ["in_stock:true"],
  "page": 1,
  "result_count": 0
}

// click event, same query_id
{
  "query_id": "3f9c0b7e-6d2a-4c1e-9a55-2b8f1d7e4a10",
  "client_id": "c-7b1e44",
  "event_attributes": { "position": 3 },
  "object": { "object_id": "SKU-100234", "object_id_field": "sku" }
}

The query_id and client_id are load-bearing. Without them you cannot connect a click to the search that produced it, or see whether a shopper refined the same question. The filter and page fields show whether a zero-result search was over-filtered, which is often the real problem. Position is what makes click data usable for the bias correction later in this article. Store the two event types in separate tables and keep raw query text in its own column, so that it can be deleted on a schedule without losing the aggregates.

Which five metrics should you track?

Five metrics cover most of the loop. The definitions follow the vendor wording, so your numbers can be compared with documentation, and each row names the event field behind it, so that a moving number can be traced to its events. Algolia’s analytics concepts list the same family of views, including searches without results and searches without clicks, which is where the weekly review starts.

The targets are where public evidence is thin. The one calibration point here is vendor guidance from Hello Retail, published in February 2026, which puts the average zero-result rate at 10–15% of searches and a target below 5%. It is a vendor blog post, not a study, so use it as a line to test against rather than a goal. For click-through and no-click rates there is no reliable public average. Set your own baseline from four to six weeks of logs, and change it only when a change has been tested.

Why does raw CTR by position mislead?

Raw click position mostly describes where results were, not how good they were. Shoppers click the top result more often because it sits on top. Google’s WSDM 2018 paper on position bias says that estimating those propensities by randomising results is “not desired because result randomization can negatively impact users’ search experience”, and proposes estimating them from regular click data instead (Google Research).

eBay’s SPIRE 2019 work goes further and estimates click propensities directly from e-commerce query–document pairs that appear at different ranks, with no live intervention and no relevance model, up to rank 500 (arXiv). Most shops will not build that. The practical rule is simpler: do not read a CTR-by-position chart as a relevance chart. Use it to see where clicks concentrate, and judge ranking changes with a method that does not depend on the position curve.

Interleaving is that method. It mixes the results of two rankers into one list and credits each click to the ranker that supplied the clicked result. Chapelle, Joachims, Radlinski and Yue reported that interleaving reduced the number of queries needed by over an order of magnitude compared with click-based signals, and that one judged query–document pair gave about as much sensitivity as one or two queries with clicks (Chapelle et al., TOIS 2012). Plain A/B testing still answers business questions such as conversion and revenue, and it is the right tool for large changes. For ranking changes, interleaving reaches an answer with far less traffic.

How do you group zero-result queries?

Group them in stages, cheapest first. Rules catch the obvious cases, such as a misspelled brand name or a SKU fragment. Normalisation handles case, separators, unit notation and plural forms. The queries that remain go through an embedding model and a clustering step. BERTopic describes the standard recipe: embed the documents with a pretrained transformer, cluster the embeddings, and label the topics with class-based TF-IDF (arXiv). A language model then names each cluster and assigns it a label.

The label vocabulary matters more than the tooling. Amazon’s Shopping Queries Dataset judges query–product pairs as Exact, Substitute, Complement or Irrelevant. It covers about 130,000 unique queries and 2.6 million manual judgements in English, Japanese and Spanish, and it was released for KDD Cup 2022 (arXiv). You do not need the data to borrow the vocabulary. For triage, a zero-result cluster usually reads as one of three gaps: a catalogue gap, where the product is not in the shop; a vocabulary gap, where the product exists under other words; or an intent gap, where the shopper asks for something the shop does not frame as a product, such as a compatibility question.

The labelling step is cheap by design. It sees cluster names and a few example queries rather than every search, so its cost follows the size of the cluster summary, not the shop’s traffic. When the weekly export has only a few dozen distinct zero-result queries, read them by hand. Clustering pays off when the list is long.

What does the weekly review look like?

The loop runs once a week and ends in one change. Export the week’s zero-result and no-click queries. Cluster and label them. Decide which gap each cluster is. Ship one fix, record it in a change log, and measure the following week, or test the fix if it is risky. Closing the loop with one change at a time keeps the attribution clean.

The fixes fall into four groups, and the gap decides which one to use.

• Synonym for a vocabulary gap. Algolia’s documented workflow starts from search analytics and adds synonyms of several kinds: regular, one-way, alternative corrections and placeholders. Synonyms do not cascade, and the documentation warns against overusing them (Algolia).
• Redirect for a query with one clear destination, such as a category page or a campaign landing page.
• Boost or pin for a product that already matches the query but ranks below weaker results.
• Data fix for a catalogue gap. If the product exists but its name or attributes do not carry the word the customer used, the fix belongs to the catalogue owner. Catalogue data quality covers how to route that work.

Clusters that are really navigation problems, such as a missing category link, are not search problems. Route them out of the loop and record them as rejected, so the same cluster does not come back every week.

The experiment step closes the circuit. Ship one change at a time, log its date and the cluster it targeted, and read the next week’s numbers against that date. Use an A/B test for a large change with enough traffic. Use interleaving for ranking changes where you need an answer sooner. The rewriting mechanics behind many of these fixes are in conversational search query rewriting; the weekly review is how you find which rewrites are worth making.

How long should search logs be kept?

Keep only the fields the loop uses, and only for as long as the loop needs them. Under the GDPR, data minimisation (Article 5(1)(c)) and storage limitation (Article 5(1)(e)) apply to search logs as they do to any personal data, and Article 32 lists pseudonymisation and encryption among the security measures (EUR-Lex).

In practice that means pseudonymous session identifiers instead of account numbers, raw query text in its own table with its own retention window, and no IP addresses after ingestion. The default matters here. Algolia uses IP addresses to tell users apart unless you pass a userToken, and it warns that backend searches otherwise group every user under the server’s IP (Algolia). An IP address is personal data you would rather replace with a pseudonymous identifier before it reaches the log.

If logs can reach named employees, the question changes. In Austria, § 96 Abs. 1 Z 3 of the Arbeitsverfassungsgesetz makes the introduction of control measures and technical systems for monitoring employees subject to works-council consent where human dignity is affected (RIS). An internal search log that can be traced to a named employee is the case to check first, and the safer design keeps the join key pseudonymous and the reporting aggregated.

Retention periods are a policy decision for your business, not a vendor default. Write the window into the schema documentation and enforce it with a scheduled delete, so the weekly export keeps working without the raw history growing forever. If you build this for a B2B shop, my B2B e-commerce work covers the catalogue side as well as the search side.

Sources

1. Algolia: search analytics concepts (metrics and definitions)
2. Algolia: query aggregation and userToken
3. User Behavior Insights (UBI) specification 1.3.0
4. Google Research: position bias estimation for unbiased learning to rank in personal search (WSDM 2018)
5. eBay: click propensity estimation for e-commerce search (SPIRE 2019)
6. Chapelle, Joachims, Radlinski, Yue: interleaved search evaluation (TOIS 2012)
7. Shopping Queries Dataset (ESCI), KDD Cup 2022
8. BERTopic: neural topic modeling with class-based TF-IDF
9. Hello Retail: ecommerce search statistics (24 Feb 2026)
10. Algolia: adding synonyms
11. Regulation (EU) 2016/679 (GDPR), EUR-Lex
12. Arbeitsverfassungsgesetz § 96, RIS (consolidated text)