Choosing an embedding model in 2026: leaderboards, costs and a bake-off on your own data

The embedding model is the least visible part of a RAG system and one of the hardest to change. Every chunk in your index was produced by it, every query must go through the same model, and vectors from a different model are not comparable. Pick badly and you find out months later, when fixing it means re-embedding the whole corpus.

This guide is how I choose one in October 2026: what the leaderboards are good for, which models belong on a shortlist, which properties matter more than rank, and how to run a small bake-off on your own data. It complements the step-by-step production RAG pipeline, which assumes the model is already chosen.

Why is the embedding model a long-term decision?

An embedding is a position in a space that only one model defines. Two models can both output 1,024 numbers per text and still be meaningless to each other, because each learned its own geometry. Google states it plainly for its own models: the spaces of gemini-embedding-001 and gemini-embedding-2 are incompatible, and moving between them requires re-embedding all existing data.

The token bill for re-embedding is rarely the problem. A corpus of 50 million tokens costs $6.50 to embed once with text-embedding-3-large at $0.13 per million. The real cost is the pipeline time: re-parsing, re-chunking, rebuilding indexes, re-running evaluations and switching traffic without downtime. That is why the choice deserves a week of measurement rather than a glance at a leaderboard.

What do MTEB and RTEB actually tell you?

MTEB is the standard public benchmark for embedding models. Its multilingual expansion, MMTEB, covers more than 500 quality-controlled tasks in over 250 languages, including retrieval, classification, clustering, instruction following, long documents and code. One finding from the MMTEB paper is worth remembering: models with billions of parameters lead on some subsets, but the best-performing publicly available model in that study was multilingual-e5-large-instruct with only 560 million parameters. Size is not the same as fitness for your task.

The weakness of a public benchmark is that it is public. When test sets overlap with training data, models score well without generalizing, which the Hugging Face team calls teaching to the test. Their answer, RTEB, launched in beta on 1 October 2025: a retrieval benchmark with open datasets plus private ones that only the MTEB maintainers evaluate, covering law, healthcare, code and finance in 20 languages, ranked by nDCG@10.

Vendor benchmarks are a third category and need the most care. Voyage reports that voyage-4-large beats OpenAI's text-embedding-3-large by 14.05% and Gemini Embedding 001 by 3.87% on average across RTEB datasets, measured by Voyage itself. Such numbers are useful to decide what to test, not what to buy.

Which models belong on a 2026 shortlist?

The table lists models I would consider for a new RAG project today, with figures from each vendor's documentation or model card as of October 2026. Prices are per million input tokens at standard rates.

Two details from the model cards matter for German or Hungarian content. Qwen3-Embedding-8B ranked first on the MTEB multilingual leaderboard on 5 June 2025 with a score of 70.58, and its authors recommend writing task instructions in English even for multilingual use. Google describes EmbeddingGemma as the highest-ranking open multilingual text embedding model under 500 million parameters on MTEB, with a context of only about 2,000 tokens. My notes on multilingual RAG for German and Hungarian cover what else changes for those languages.

Which criteria matter more than leaderboard rank?

A two-point difference on an aggregate benchmark disappears next to a mismatch in any of these properties:

• Languages and domain. Check the languages your users actually query in, including mixed-language queries, and whether the model saw text like yours: part numbers, legal German, source code.
• Context length versus chunk size. A 2,000-token model is fine for 400-token chunks; it truncates a 6,000-token section silently. Voyage truncates over-length input by default unless you set truncation to false.
• Dimensions and output types. Matryoshka models let you choose a smaller vector at request time, and several APIs return int8 or binary vectors directly. Both cut index memory, as described in embedding quantization.
• Hosting and data residency. Whether your documents may leave your infrastructure, and in which region, decides between a hosted API and open weights before quality does.
• Query latency and volume. Every search embeds a query first. A network round trip per query is acceptable for a chat assistant and noticeable in search-as-you-type.
• Family and longevity. A model family with a shared space, a published deprecation policy and several sizes gives you more room to adapt later than a single checkpoint.

Hosted API or open weights?

Hosted models are the fastest start: no GPUs, no serving stack, and batch pricing for the bulk indexing run. Google offers its embedding models through the Batch API at 50% of the standard price, and Voyage bills batch requests 33% below its standard rates. The trade-off is that every document and every query passes through a third party, which turns the choice into a data-protection question. I covered region controls and retention options in GDPR and LLM data residency.

Open-weight models move the work to you, but they have become small. EmbeddingGemma has about 308 million parameters and stays under 200 MB of RAM once quantized; the Qwen3-Embedding series starts at 0.6 billion parameters under Apache 2.0; Voyage published voyage-4-nano on Hugging Face under Apache 2.0. On an ordinary CPU server these handle query embedding comfortably, and the bulk indexing run can be a one-off job on a rented GPU. Even if you end up on a hosted model, one open model in the bake-off tells you what the hosted premium buys.

What do shared embedding spaces change?

The newest shift is model families trained into one space. Voyage's four Voyage 4 models produce compatible embeddings, so documents embedded once with voyage-4-large can be searched with queries from voyage-4-lite or the open voyage-4-nano. Voyage calls this asymmetric retrieval. Index quality comes from the expensive model, while the per-query cost and latency come from the cheap one, and you can change the query model later without touching the index.

The compatibility Voyage documents covers the four Voyage 4 models and nothing earlier, so plan a re-embed when moving from the Voyage 3 generation. A shared space is a reason to prefer a family, not a reason to stop storing which model produced each vector.

Which API details cause silent quality loss?

Most bad bake-off results I see come from calling a model wrong, not from the model. Three details are worth checking against the docs for every candidate:

1. Query and document modes. Voyage prepends a different prompt for input_type query and document. Gemini's gemini-embedding-001 takes a task_type such as RETRIEVAL_QUERY, while gemini-embedding-2 expects a task prefix in the text. Qwen3-Embedding expects an instruction on the query side, which its authors say typically improves results by 1 to 5%.
2. Normalization after truncation. OpenAI embeddings are normalized to length 1, and the dimensions parameter keeps them normalized. If you cut a vector yourself, or use gemini-embedding-001 below 3,072 dimensions, you must normalize it again before using dot product.
3. Truncation of long inputs. Know whether the API cuts or rejects over-length input, and log the token count per chunk so a chunker change does not quietly push chunks past the limit.

import numpy as np

def prepare(vec: list[float], dims: int | None = None) -> np.ndarray:
    """Truncate a Matryoshka embedding and re-normalize it for dot-product search."""
    v = np.asarray(vec, dtype=np.float32)
    if dims is not None:
        v = v[:dims]               # keep the leading dimensions only
    return v / np.linalg.norm(v)   # unit length again, so dot product == cosine

# Store the model id next to every vector: vectors from different models never mix.
row = {"chunk_id": "doc-42#3", "model": "voyage-4-large", "dims": 1024,
       "embedding": prepare(raw_vector, 1024).tolist()}

How do you run a bake-off on your own data?

A useful bake-off is small and takes about a week. You need real queries, the chunks that answer them, and the same chunking for every candidate so that only the model varies.

1. Collect 100 to 300 real queries from search logs, support tickets or users, and label the chunks that answer each one. Include short keyword queries and long questions, and every language your users write in.
2. Pick three to five candidates: one cheap hosted model, one strong hosted model, one open-weight model, and whatever your current system uses.
3. Embed the same chunks with each, using the correct query and document modes, at the dimension you would deploy.
4. Measure recall@10 and nDCG@10 per candidate, then repeat with your hybrid search and reranker on top, since a reranker can hide first-stage differences. The metrics are explained in Evaluating RAG.
5. Put quality, index size, cost per million tokens and p95 query latency in one table, and choose the cheapest model within your quality budget.

How do you plan for the next model?

You will switch again; the release pace of the past two years makes that a safe bet. Design for it from day one: keep the parsed and chunked text so re-embedding does not mean re-parsing, version the index name with the model, and build the new index next to the old one. Run the bake-off queries against both, shadow a share of live traffic, then switch with a configuration change and keep the old index until the evaluation holds. The same discipline applies to the generation model, which I described in LLM model migrations without regressions.

What are sensible defaults in late 2026?

These are my opinions, not benchmark results. For a German or Hungarian B2B knowledge base that may use a hosted API, I start the bake-off with a model family that has a shared space and multiple sizes, one Gemini model, and Qwen3-Embedding-0.6B as the open baseline. When documents must stay on-premises, I compare Qwen3-Embedding at two sizes against EmbeddingGemma and accept the larger one only if the measured gain justifies the extra hardware. In both cases I deploy at 1,024 dimensions or fewer unless the bake-off shows a clear loss.

The model matters less than people expect once chunking, hybrid search and reranking are in place. Spend the first week on the evaluation set, because it outlives every model you will try.

Sources

1. Enevoldsen et al.: MMTEB, Massive Multilingual Text Embedding Benchmark (2025)
2. Hugging Face: Introducing RTEB, a new standard for retrieval evaluation (Oct 2025)
3. OpenAI: Vector embeddings guide
4. OpenAI: API pricing
5. Voyage AI: The Voyage 4 model family, shared embedding space with MoE architecture (Jan 2026)
6. Voyage AI: Text embeddings documentation
7. Voyage AI: Pricing
8. Google: Gemini API embeddings documentation
9. Google: Gemini API pricing
10. Cohere: Embed models documentation
11. Qwen: Qwen3-Embedding-8B model card
12. Google Developers Blog: Introducing EmbeddingGemma (Sep 2025)