SPLADE and learned sparse retrieval in practice: BM25, index size, latency

Learned sparse retrieval keeps the machinery of BM25, an inverted index of vocabulary terms, but replaces hand-tuned term statistics with weights learned by a language model and adds expansion terms that never appear in the text. SPLADE is the best-known family. A document about a “stainless ball valve” can then match a query for a “corrosion resistant shut-off valve”, while the index, the query engine and exact-term matching stay the same ones you already run.

This article covers how SPLADE expands text, the model family and its license trap, what the index and queries cost, how Elasticsearch and OpenSearch run it, and when it beats BM25 or a dense hybrid. Where sparse retrieval fits in a full RAG pipeline is covered in RAG pipelines: chunking, hybrid search and reranking; numbers are as of October 2026.

What learned sparse retrieval is

It is BM25-shaped retrieval with learned weights and learned vocabulary expansion. Each document and query becomes a vector over the model’s vocabulary, mostly zeros, with positive weights on the terms that matter, including related terms the model adds. Because the vector is sparse and keyed by terms, it fits an inverted index and keeps the explainability of keyword search: you can see which terms matched.

The gains show up mostly out of domain. In the SPLADE++ paper, on 13 BEIR datasets, BM25 averaged 43.7 nDCG@10, Contriever 47.5, ColBERTv2 49.7 and SPLADE++ 50.5 to 50.7; summing SPLADE++ and BM25 scores reached 52.1, so the two remain complementary (SPLADE++). In domain, on MS MARCO, the same paper reports BM25 at 18.4 MRR@10 against 38.0 for SPLADE++.

Dense retrievers solve a similar problem in a different way: they map text to a few hundred or thousand continuous dimensions and match by vector similarity. They are strong on meaning and weak on rare exact terms, and their results are hard to explain. Learned sparse retrieval sits between the two: it keeps exact-term matching and an inspectable vocabulary, and borrows the language model only to decide which terms matter and which related terms to add.

How SPLADE expands text

SPLADE reuses the masked-language-model head of BERT, which already predicts, for every input token, how likely every vocabulary term is in that position. It passes those scores through a ReLU and a log saturation, max-pools them over the input tokens, and gets one weight per vocabulary term, 30,522 of them for the BERT vocabulary (SPLADE-v3). A sparsity regularizer during training pushes most weights to zero, so each text keeps only its important terms and a limited set of expansions.

Expansion is also where things go wrong. A model trained on web questions may add terms that are plausible in general English but misleading in your domain, for example mapping a product line name to an unrelated common word. Printing the top expansion terms for a sample of your own documents, as in the snippet below, takes minutes and shows quickly whether the model understands your vocabulary or needs adaptation.

The regularizer strength is the main dial: stronger regularization means fewer non-zero terms, a smaller index and faster queries, at some quality cost. Naver’s own repository describes SPLADE as “more a class of models rather than a model per se” for exactly this reason (naver/splade). Query and document can also be encoded differently, which is what the efficient and doc-only variants below exploit.

# Encode with a learned sparse model and inspect the expansion terms (sentence-transformers SparseEncoder)
from sentence_transformers import SparseEncoder

model = SparseEncoder("Qdrant/Splade_PP_en_v1")        # Apache-2.0 re-host; check the license you deploy
doc = model.encode_document(["Stainless steel ball valve DN50, PN40, threaded ends"])
query = model.encode_query(["corrosion resistant shut-off valve 2 inch"])

for token, weight in model.decode(doc[0], top_k=12):    # expansion terms that were never in the text
    print(f"{token:>14}  {weight:.2f}")
print(model.similarity(query, doc))                     # dot product over the shared vocabulary

The model family and its license trap

SPLADE v2 was followed by SPLADE++, which added distillation from a cross-encoder, mined hard negatives and CoCondenser initialization, and by SPLADE-v3 in March 2024, which reports 40.2 MRR@10 on MS MARCO and 51.7 average nDCG@10 on BEIR-13 with a 512-token maximum input (SPLADE-v3). In 2026 Naver also released SPLADE-Code-06B, a Qwen3-based 0.6B model for code that its card presents as the top MTEB performer below 1B parameters; treat that as the vendor’s claim (SPLADE-Code).

The efficiency reality: index size and latency

Expansion makes posting lists longer, so a learned sparse index is larger and slower to scan than a BM25 index over the same text; how much depends on the regularization strength of the model, and I found no single primary number for the ratio. The measured costs are concrete, though. OpenSearch’s benchmark on MS MARCO put BM25 at 5.4 ms fixed latency plus 2.80 ms per million documents, and doc-only neural sparse at 8.7 ms plus 5.15 ms per million, about 1.8 times BM25’s scan (OpenSearch).

The expensive mode is running a model on every query. In the same benchmark, bi-encoder mode added 265.4 ms of fixed latency and 20.58 ms per million documents; OpenSearch’s two-phase processor, which first scores with high-weight tokens and then refines, cut average latency from 53.56 to 38.61 ms in doc-only mode and from 300.79 to 121.64 ms in bi-encoder mode. Naver’s efficiency work reached latency within 4 ms of BM25 with less than 10 percent MRR@10 loss using separate, lighter query encoders (naver/splade).

Pruning is the cheapest lever. Static pruning of learned sparse indexes gave a 2x speedup for at most 2 percent effectiveness loss, or 4x for at most 8 percent, and rerankers stayed robust on candidates from pruned indexes (static pruning). OpenSearch’s ingest processor supports several prune types, and max_ratio at 0.1 cuts storage by about 40 percent for less than 1 percent relevance loss (OpenSearch).

Engine support in Elasticsearch and OpenSearch

Both engines store learned sparse vectors as weighted terms on the inverted index. In Elasticsearch, sparse_vector is the field type to use; token pruning is on by default since 9.1, sparse vectors are excluded from _source by default since 9.2, and weights are stored with 9 significant bits, about 0.4 percent relative error (Elastic). Elastic’s own model, ELSER v2, is English-only, encodes the first 512 tokens of a field and won 10 of 12 BEIR datasets against BM25 with an average gain of 18 percent nDCG@10 (ELSER).

ELSER’s query-side optimisation is worth copying conceptually: prune query tokens that are much more frequent than expected, keep tokens above 0.4 of the maximum weight, and rescore the top 50 with the full token set. Elastic reports an average 3.4x speedup for a negligible change in nDCG@10 (Elastic Search Labs). For fusion, Elasticsearch’s RRF retriever combines a BM25 standard retriever with an ELSER retriever using a default rank_constant of 60 (RRF). OpenSearch offers doc-only and bi-encoder neural sparse modes, a two-phase processor, and since 3.3 approximate search on sparse_vector fields.

Dedicated vector stores support sparse vectors too, which matters if you already run one; the Qdrant review covers that side. The general mechanics of running vectors inside a search cluster are in vector search in Elasticsearch and OpenSearch.

In practice the choice of engine follows the cluster you already run. If you are on Elasticsearch with an Elastic subscription and English content, ELSER is the least work. If you are on OpenSearch, its own sparse models and the doc-only mode are the natural start. If you want an open SPLADE checkpoint in either engine, encode documents yourself in an ingest job and write the token weights into a sparse field, which both engines accept.

When it beats BM25, and when it does not

Learned sparse retrieval helps most when queries and documents use different words for the same thing: user questions against technical documentation, natural-language product searches against terse catalog text, synonyms and paraphrases in English. It also gives you a second retriever whose results you can explain term by term, which dense vectors cannot offer. In B2B catalogs from Spryker and Pimcore work, full of article numbers and normalized attributes, that combination of expansion and exact terms is the main motivation.

It helps less when queries are mostly exact identifiers that BM25 already matches, when the content is not English and the model is English-only, as ELSER is, or when the corpus is so large that a second, longer posting list is the expensive part. It is not uniformly better on benchmarks either; the SPLADE++ results show BM25 still winning on some BEIR sets. Product search with part numbers specifically is covered in semantic product search for B2B.

Training and domain adaptation

Adapting a sparse model is within reach of one GPU. The naver/splade repository ships configurations per training scenario, including a single-GPU configuration on a 16 GB card that reaches 37.2 MRR@10 (naver/splade). Sentence Transformers now has a first-class SparseEncoder with SPLADE-specific losses such as SpladeLoss, CachedSpladeLoss and FlopsLoss, and supports asymmetric query and document encoding (Sentence Transformers).

For languages other than English, adaptation is often the only path, because most published sparse checkpoints, ELSER among them, are English-only. A multilingual base model with a BERT-style vocabulary can be trained into a sparse encoder with the same losses, but the vocabulary itself limits what can be expanded; German compounds, for example, are split into word pieces, and the expansion works on those pieces rather than on whole words.

The data side is the same as for dense fine-tuning: query-document pairs from logs or generated queries, mined hard negatives and a frozen evaluation set; fine-tuning embedding models covers that loop. The sparsity regularizer is the extra knob, and it is worth sweeping, because it trades quality against index size and latency directly.

A practical checklist

1. Measure BM25 and your current hybrid on a labelled query set first.
2. Check the license of the exact checkpoint; Naver’s are non-commercial.
3. Check the model’s language coverage against your content.
4. Start in doc-only mode and prune tokens before measuring latency.
5. Fuse with BM25 or dense results through RRF, then compare against fusion without it.
6. Measure index size and p95 latency on your real document count.
7. Keep it only if recall at your k improves enough to pay for the second index.

If you are deciding whether a learned sparse retriever earns its place next to BM25 and dense search, see my AI engineering work.

Sources

1. naver/splade on GitHub
2. SPLADE++ (arXiv 2205.04733)
3. naver/splade-v3 model card
4. Static pruning for learned sparse indexes (arXiv 2304.12702)
5. OpenSearch blog: faster semantic sparse retrieval in 2.12
6. OpenSearch docs: sparse encoding processor
7. Elastic docs: sparse_vector field type
8. Elastic docs: ELSER
9. Elastic Search Labs: ELSER v2, part 2
10. Sentence Transformers: sparse encoder usage