Cross-encoder rerankers compared: Cohere, bge, Voyage, Jina and late interaction (2026)

A reranker reads a query and a candidate document together and returns a relevance score, which makes it more accurate than comparing two embeddings and far too slow to run on a whole corpus. That is why retrieval is built in two stages: a fast retriever finds 50 to 250 candidates, and a cross-encoder reorders them so the best few reach the model. The decision is which reranker, how many candidates, and whether the gain is worth the latency.

This article compares the scoring architectures, the models available as of October 2026, hosted prices and self-hosting, late interaction and LLM rerankers as alternatives, and a small evaluation to choose k. Where reranking sits in the full pipeline is covered in RAG pipelines: chunking, hybrid search and reranking.

What a reranker buys you, and what it costs

A reranker buys precision at the top of the list and costs latency proportional to k. The reason it cannot replace retrieval is arithmetic: the Sentence-BERT paper notes that finding the most similar pair in 10,000 sentences takes about 50 million BERT inference computations, roughly 65 hours, which “makes it unsuitable for semantic similarity search” (SBERT). The documented production shape is therefore a bi-encoder that retrieves candidates and a cross-encoder that re-ranks them (Sentence Transformers).

The two numbers to know before adding one are the recall of your first stage at k and the added latency of scoring k pairs. If the right document is not in the top 100, no reranker will find it; if it is at position 30 and you show five results, a reranker is exactly the fix. Product search with part numbers and long German compound descriptions is a typical case where the right item is retrieved but ranked too low, see semantic product search for B2B.

The second number matters as much as the first. Every candidate is one more forward pass for a pointwise model, so going from 50 to 200 candidates roughly quadruples the reranking time. In an interactive search box the user notices that; in a background RAG answer that already waits seconds for generation, it may not matter at all. Write the latency budget down before you compare models, because it decides which half of the table below is even eligible.

How rerankers score: pairs, lists and token vectors

There are three architectures, plus fusion without a model. A pointwise cross-encoder concatenates the query and one document and outputs one relevance logit, so k candidates mean k forward passes. A listwise reranker puts the query and all candidates into one context window and orders them in a single pass. Late interaction stores a vector per token at index time and matches query tokens against them with MaxSim, which moves most of the work out of the query path.

Jina Reranker v3 is the clearest listwise example: a 0.6B model that reads the query and every candidate in one window, described by Jina as “last but not late interaction.” Its v3.5 release in August 2026 keeps 0.6B parameters, extends the context to 131K tokens and reports reranking up to 1.56 times faster than v3 (Jina). Listwise models see the candidates relative to each other, which helps with near-duplicates, but the whole list has to fit in context.

The 2026 line-up

As of October 2026 the hosted lines are Cohere Rerank 4, released in December 2025 as rerank-v4.0-pro and rerank-v4.0-fast with a 32k context, and Voyage rerank-3 and rerank-3-lite, released on 30 September 2026, also with 32k. Voyage reports rerank-3 at 2.72 percent above Cohere Rerank 4 Pro across 95 datasets (Voyage); that is a vendor claim on a vendor suite, not an independent result. The open models are licensed for commercial self-hosting.

BAAI/bge-reranker-v2-m3 is the common open default: a 0.6B multilingual cross-encoder on an XLM-RoBERTa base, Apache-2.0, with more than 15 million downloads in the last month (Hugging Face). Qwen3-Reranker-8B is the heavyweight open option from the Qwen3 Embedding series, and mixedbread’s mxbai-rerank-large-v2 sits between them at 1.54B parameters, about 3.09 GB in FP16, covering more than 100 languages (mixedbread).

Two practical differences matter more than a point of nDCG. Context length decides whether a long product datasheet or contract clause fits as one document or has to be split; Cohere’s v4 models split documents at 32,764 tokens and keep the highest chunk score. Language coverage decides whether German and Hungarian queries work as well as English ones, which is why the multilingual bge, mxbai and Cohere models are the usual candidates for European content.

Prices: per search, per token, or your own CPU

Hosted rerankers are priced in two different units, which makes them hard to compare at a glance. Cohere charges per search: $2.00 per 1,000 for Rerank 4 Fast and $2.50 for Rerank 4 Pro, where a search is one query with up to 100 documents and documents over 500 tokens count as several (Cohere pricing). Voyage charges per token: $0.05 per million for rerank-3 and $0.02 for rerank-3-lite (Voyage pricing).

Turn both into the same arithmetic before choosing. With 100 candidates of about 300 tokens each, one query is roughly 30,000 tokens: about $0.0015 on rerank-3 and $0.0025 on Rerank 4 Pro. The ratio flips with longer chunks, because Cohere counts per document and Voyage per token. Cohere also documents limits worth knowing: a 2,048-token query cap and a recommendation against sending more than 1,000 documents per request (Cohere).

Self-hosting turns the price into compute. On CPU, size is the practical limit: FastEmbed’s ONNX rerankers range from 0.08 GB for ms-marco-MiniLM-L-6-v2 to 1.11 GB for jina-reranker-v2-base-multilingual, which is licensed for non-commercial use only (FastEmbed). I found no primary benchmark for per-pair CPU latency, so measure on your own hardware with your own chunk length before committing. On my own Nuxt site, where I run LLM features, the cheap path for this kind of problem is hybrid retrieval plus a small ONNX model on the server, not an LLM call per query.

Late interaction as an alternative

Late interaction is a different trade-off rather than a better cross-encoder. ColBERTv2 keeps a vector per token, uses residual compression to cut the storage footprint of late-interaction models by six to ten times, and scores with cheap MaxSim at query time (ColBERTv2). You pay in index size instead of query latency, which suits large k and high query volume. For page images, the same idea drives ColPali, covered in multimodal RAG with ColPali. The rerankers library wraps cross-encoders, APIs and late-interaction models behind one interface, which makes swapping them in an eval easy (rerankers).

LLM rerankers, and when RRF alone is enough

A general LLM can rerank, but it is rarely the cheapest way. RankGPT showed that properly instructed GPT-4 can be competitive with, or better than, supervised rerankers on IR benchmarks, and that a distilled 440M model beat a 3B supervised model on BEIR (RankGPT). Voyage’s own study argues the other side, reporting purpose-built rerankers as up to 60 times cheaper and 48 times faster than LLMs (Voyage); both sources have a stake, so test on your data.

Reciprocal rank fusion is the cheapest step and often the first one to add. It merges ranked lists from several retrievers using only their positions; Elasticsearch implements it with a rank_constant that defaults to 60 and a rank_window_size (Elastic). RRF is enough when BM25 and vector search each find different good results and the top of the fused list is already right. Add a model on top when the right document is consistently in the fused top 50 but not in the top 5.

Picking k and the model with a small eval

Start from a known default and let a small labelled set move it. Azure AI Search’s semantic ranker takes only the top 50 results forward and uses up to 2,000 tokens per document, starting from a BM25 or RRF result set (Azure). That is a sensible starting k. Then label 50 real queries with the documents that should answer them and measure recall at 10 before and after reranking for k of 25, 50 and 100.

# A small reranker eval: 50 labelled queries, recall@k before and after reranking
from sentence_transformers import CrossEncoder

reranker = CrossEncoder("BAAI/bge-reranker-v2-m3")

def recall_at(ranked_ids, relevant_ids, n):
    return len(set(ranked_ids[:n]) & relevant_ids) / len(relevant_ids)

for k in (25, 50, 100):
    before, after = [], []
    for q in labelled_queries:                       # [{"text": ..., "relevant": {...}}]
        hits = first_stage(q["text"], k=k)           # your hybrid search: [(id, text), ...]
        scores = reranker.predict([(q["text"], t) for _, t in hits])
        reranked = [i for (i, _), _ in sorted(zip(hits, scores), key=lambda p: -p[1])]
        before.append(recall_at([i for i, _ in hits], q["relevant"], 10))
        after.append(recall_at(reranked, q["relevant"], 10))
    print(k, sum(before) / len(before), sum(after) / len(after))

Stop increasing k when recall at 10 stops improving, and weigh that against the latency you measured. Run the same script with two or three models and keep the cheapest one within a point of the best. Re-run it after every embedding model change, because the first stage decides what the reranker can see. How to keep such evals in a release process is covered in LLM evals for product features.

When not to add a reranker, and a checklist

Skip the reranker when the first stage already puts the right answer in the top few, when exact identifiers dominate the queries and BM25 finds them, or when the latency budget cannot absorb k extra forward passes. In those cases better chunking, metadata filters or RRF give more for less. Add one when the eval shows relevant documents sitting just below the cut.

1. Measure first-stage recall at k before choosing any reranker.
2. Fuse BM25 and vector results with RRF before adding a model.
3. Start with k of 50 and return 5 to 20 results.
4. Convert hosted prices to cost per query at your chunk length.
5. Check the license before self-hosting; some rerankers are non-commercial.
6. Measure added latency on your own hardware, not from published numbers.
7. Re-run the 50-query eval after every model or chunking change.

If you are tuning retrieval for a RAG system or a product search and want the reranker decision backed by numbers, see my AI engineering work. Vector store choices that sit under all of this are covered in the Qdrant review.

Sources

1. Sentence-BERT (arXiv 1908.10084)
2. Sentence Transformers: retrieve and re-rank
3. BAAI/bge-reranker-v2-m3 model card
4. Cohere docs: reranking best practices
5. Cohere pricing
6. Voyage AI blog: rerank-3 (30 Sep 2026)
7. Voyage AI docs: pricing
8. Jina AI: Reranker v3.5 (3 Aug 2026)
9. ColBERTv2 (arXiv 2112.01488)
10. Elasticsearch docs: reciprocal rank fusion
11. Azure AI Search: semantic ranking overview
12. RankGPT (arXiv 2304.09542)