[{"data":1,"prerenderedAt":711},["ShallowReactive",2],{"blog-splade-learned-sparse-retrieval-en":3},{"slug":4,"published":5,"updated":6,"minutes":7,"category":8,"tags":9,"keywords":15,"about":25,"sources":35,"cover":66,"og":67,"expertise":68,"locales":69,"lang":70,"title":73,"description":74,"coverAlt":75,"metaTitle":76,"takeaways":77,"faq":83,"toc":99,"blocks":127,"others":457},"splade-learned-sparse-retrieval","2026-09-28","2026-10-01",8,"rag",[10,11,12,13,14],"SPLADE","Sparse retrieval","BM25","Hybrid search","Elasticsearch",[16,10,17,18,19,20,21,22,23,24],"learned sparse retrieval","SPLADE++ vs BM25","neural sparse search","Elasticsearch sparse_vector","ELSER","rank_features","OpenSearch neural sparse","learned sparse retrieval index size","sentence-transformers sparse encoder",[26,29,32],{"name":27,"url":28},"Okapi BM25","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FOkapi_BM25",{"name":30,"url":31},"Inverted index","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FInverted_index",{"name":33,"url":34},"BERT (language model)","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FBERT_%28language_model%29",[36,39,42,45,48,51,54,57,60,63],{"title":37,"url":38},"naver\u002Fsplade on GitHub","https:\u002F\u002Fgithub.com\u002Fnaver\u002Fsplade",{"title":40,"url":41},"SPLADE++ (arXiv 2205.04733)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2205.04733",{"title":43,"url":44},"naver\u002Fsplade-v3 model card","https:\u002F\u002Fhuggingface.co\u002Fnaver\u002Fsplade-v3",{"title":46,"url":47},"Static pruning for learned sparse indexes (arXiv 2304.12702)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.12702",{"title":49,"url":50},"OpenSearch blog: faster semantic sparse retrieval in 2.12","https:\u002F\u002Fopensearch.org\u002Fblog\u002FA-deep-dive-into-faster-semantic-sparse-retrieval-in-os-2-12\u002F",{"title":52,"url":53},"OpenSearch docs: sparse encoding processor","https:\u002F\u002Fdocs.opensearch.org\u002Flatest\u002Fingest-pipelines\u002Fprocessors\u002Fsparse-encoding\u002F",{"title":55,"url":56},"Elastic docs: sparse_vector field type","https:\u002F\u002Fwww.elastic.co\u002Fdocs\u002Freference\u002Felasticsearch\u002Fmapping-reference\u002Fsparse-vector",{"title":58,"url":59},"Elastic docs: ELSER","https:\u002F\u002Fwww.elastic.co\u002Fdocs\u002Fexplore-analyze\u002Fmachine-learning\u002Fnlp\u002Fml-nlp-elser",{"title":61,"url":62},"Elastic Search Labs: ELSER v2, part 2","https:\u002F\u002Fwww.elastic.co\u002Fsearch-labs\u002Fblog\u002Fintroducing-elser-v2-part-2",{"title":64,"url":65},"Sentence Transformers: sparse encoder usage","https:\u002F\u002Fsbert.net\u002Fdocs\u002Fsparse_encoder\u002Fusage\u002Fusage.html","\u002Fimages\u002Fblog\u002Fsplade-learned-sparse-retrieval\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fsplade-learned-sparse-retrieval\u002Fog.jpg","ai-engineer",[70,71,72],"en","de","hu","SPLADE and learned sparse retrieval in practice: BM25, index size, latency","Learned sparse retrieval in practice: how SPLADE expands terms, models and licenses, index size and latency, engine support and when it beats BM25.","A word strip fans out into many thin bars, a few tall ones feeding an index box.","SPLADE and learned sparse retrieval · Balázs Csorba",[78,79,80,81,82],"Learned sparse retrieval keeps BM25’s inverted index but learns term weights and adds expansion terms, so documents match queries that use different words.","On the 13-dataset BEIR subset, SPLADE++ reached about 50.5 to 50.7 mean nDCG@10 against 43.7 for BM25, and summing SPLADE++ with BM25 reached 52.1, so fusion still adds.","Naver’s own SPLADE checkpoints are CC BY-NC-SA 4.0, which rules them out for commercial use; Apache-2.0 re-hosts and ELSER are the usual workarounds.","Cost is real but manageable: OpenSearch measured doc-only sparse retrieval at about 1.8 times BM25’s index scan, and static pruning gives a 2x speedup for at most 2 percent quality loss.","Both Elasticsearch and OpenSearch run learned sparse vectors on the inverted index and fuse them with BM25 or dense results; measure on your queries before adding a second retriever.",[84,87,90,93,96],{"q":85,"a":86},"What is learned sparse retrieval?","A retrieval method that represents text as a sparse vector of vocabulary terms with learned weights, including expansion terms that do not appear in the text. It runs on a normal inverted index like BM25, but matches related wording the way a dense model would.",{"q":88,"a":89},"Is SPLADE better than BM25?","On out-of-domain benchmarks, usually yes: SPLADE++ averaged about 50.7 nDCG@10 on 13 BEIR datasets against 43.7 for BM25. It is not better everywhere, and combining it with BM25 scored higher still. Test on your own queries, especially if they are dominated by exact identifiers.",{"q":91,"a":92},"Can I use SPLADE commercially?","Naver’s official SPLADE checkpoints and repository are licensed CC BY-NC-SA 4.0, which is non-commercial. Apache-2.0 community re-hosts such as Qdrant\u002FSplade_PP_en_v1 exist, and Elasticsearch’s ELSER and OpenSearch’s own sparse models are alternatives. Check the license of the exact checkpoint you deploy.",{"q":94,"a":95},"How much slower is learned sparse retrieval than BM25?","In OpenSearch’s benchmark, doc-only neural sparse search scanned the index at about 5.15 ms per million documents against 2.80 ms for BM25, about 1.8 times. Bi-encoder mode, which runs a model per query, was much slower. Pruning tokens and a two-phase query reduce the gap.",{"q":97,"a":98},"Does Elasticsearch support SPLADE-style retrieval?","Yes. Elasticsearch has a sparse_vector field type and its own learned sparse model, ELSER, which is English-only and encodes the first 512 tokens of a field. Token pruning is on by default since 9.1, and RRF can fuse ELSER with BM25 or dense retrievers.",[100,103,106,109,112,115,118,121,124],{"id":101,"title":102},"what-learned-sparse-retrieval-is","What learned sparse retrieval is",{"id":104,"title":105},"how-splade-expands","How SPLADE expands text",{"id":107,"title":108},"the-model-family-and-licenses","The model family and its license trap",{"id":110,"title":111},"the-efficiency-reality","The efficiency reality: index size and latency",{"id":113,"title":114},"engine-support","Engine support in Elasticsearch and OpenSearch",{"id":116,"title":117},"when-learned-sparse-beats-bm25","When it beats BM25, and when it does not",{"id":119,"title":120},"training-and-domain-adaptation","Training and domain adaptation",{"id":122,"title":123},"a-practical-checklist","A practical checklist",{"id":125,"title":126},"sources","Sources",[128,136,145,148,151,159,162,163,170,176,179,186,189,190,201,213,286,287,292,296,301,314,315,332,351,364,367,368,371,378,379,388,391,399,400,417,424,425],{"type":129,"content":130},"paragraph",[131,135],{"tag":132,"children":133},"strong",[134],"Learned sparse retrieval"," keeps the machinery of BM25, an inverted index of vocabulary terms, but replaces hand-tuned term statistics with weights learned by a language model and adds expansion terms that never appear in the text. SPLADE is the best-known family. A document about a “stainless ball valve” can then match a query for a “corrosion resistant shut-off valve”, while the index, the query engine and exact-term matching stay the same ones you already run.",{"type":129,"content":137},[138,139,144],"This article covers how SPLADE expands text, the model family and its license trap, what the index and queries cost, how Elasticsearch and OpenSearch run it, and when it beats BM25 or a dense hybrid. Where sparse retrieval fits in a full RAG pipeline is covered in ",{"tag":140,"to":141,"children":142},"link","\u002Fblog\u002Frag-pipeline-chunking-hybrid-search-reranking",[143],"RAG pipelines: chunking, hybrid search and reranking","; numbers are as of October 2026.",{"type":146,"level":147,"id":101,"text":102},"heading",2,{"type":129,"content":149},[150],"It is BM25-shaped retrieval with learned weights and learned vocabulary expansion. Each document and query becomes a vector over the model’s vocabulary, mostly zeros, with positive weights on the terms that matter, including related terms the model adds. Because the vector is sparse and keyed by terms, it fits an inverted index and keeps the explainability of keyword search: you can see which terms matched.",{"type":129,"content":152},[153,154,158],"The gains show up mostly out of domain. In the SPLADE++ paper, on 13 BEIR datasets, BM25 averaged 43.7 nDCG@10, Contriever 47.5, ColBERTv2 49.7 and SPLADE++ 50.5 to 50.7; summing SPLADE++ and BM25 scores reached 52.1, so the two remain complementary (",{"tag":155,"href":41,"children":156},"a",[157],"SPLADE++","). In domain, on MS MARCO, the same paper reports BM25 at 18.4 MRR@10 against 38.0 for SPLADE++.",{"type":129,"content":160},[161],"Dense retrievers solve a similar problem in a different way: they map text to a few hundred or thousand continuous dimensions and match by vector similarity. They are strong on meaning and weak on rare exact terms, and their results are hard to explain. Learned sparse retrieval sits between the two: it keeps exact-term matching and an inspectable vocabulary, and borrows the language model only to decide which terms matter and which related terms to add.",{"type":146,"level":147,"id":104,"text":105},{"type":129,"content":164},[165,166,169],"SPLADE reuses the masked-language-model head of BERT, which already predicts, for every input token, how likely every vocabulary term is in that position. It passes those scores through a ReLU and a log saturation, max-pools them over the input tokens, and gets one weight per vocabulary term, 30,522 of them for the BERT vocabulary (",{"tag":155,"href":44,"children":167},[168],"SPLADE-v3","). A sparsity regularizer during training pushes most weights to zero, so each text keeps only its important terms and a limited set of expansions.",{"type":171,"src":172,"alt":173,"wide":174,"caption":175},"image","\u002Fimages\u002Fblog\u002Fsplade-learned-sparse-retrieval\u002Fdiagram-8.webp","How SPLADE turns text into a sparse vector Five steps from left to right: the input text; a BERT masked-language-model head scores every vocabulary term for every input token; a ReLU and log saturation keep the weights positive and damped; max pooling over tokens gives one weight per vocabulary term, with a sparsity regularizer keeping most at zero; and the resulting 30,522-dimensional sparse vector is stored in a normal inverted index.",true,"The expansion terms come from the language model, but the index and the query engine are the same ones BM25 uses.",{"type":129,"content":177},[178],"Expansion is also where things go wrong. A model trained on web questions may add terms that are plausible in general English but misleading in your domain, for example mapping a product line name to an unrelated common word. Printing the top expansion terms for a sample of your own documents, as in the snippet below, takes minutes and shows quickly whether the model understands your vocabulary or needs adaptation.",{"type":129,"content":180},[181,182,185],"The regularizer strength is the main dial: stronger regularization means fewer non-zero terms, a smaller index and faster queries, at some quality cost. Naver’s own repository describes SPLADE as “more a class of models rather than a model per se” for exactly this reason (",{"tag":155,"href":38,"children":183},[184],"naver\u002Fsplade","). Query and document can also be encoded differently, which is what the efficient and doc-only variants below exploit.",{"type":187,"code":188},"code","# Encode with a learned sparse model and inspect the expansion terms (sentence-transformers SparseEncoder)\nfrom sentence_transformers import SparseEncoder\n\nmodel = SparseEncoder(\"Qdrant\u002FSplade_PP_en_v1\")        # Apache-2.0 re-host; check the license you deploy\ndoc = model.encode_document([\"Stainless steel ball valve DN50, PN40, threaded ends\"])\nquery = model.encode_query([\"corrosion resistant shut-off valve 2 inch\"])\n\nfor token, weight in model.decode(doc[0], top_k=12):    # expansion terms that were never in the text\n    print(f\"{token:>14}  {weight:.2f}\")\nprint(model.similarity(query, doc))                     # dot product over the shared vocabulary",{"type":146,"level":147,"id":107,"text":108},{"type":129,"content":191},[192,193,195,196,200],"SPLADE v2 was followed by SPLADE++, which added distillation from a cross-encoder, mined hard negatives and CoCondenser initialization, and by SPLADE-v3 in March 2024, which reports 40.2 MRR@10 on MS MARCO and 51.7 average nDCG@10 on BEIR-13 with a 512-token maximum input (",{"tag":155,"href":44,"children":194},[168],"). In 2026 Naver also released SPLADE-Code-06B, a Qwen3-based 0.6B model for code that its card presents as the top MTEB performer below 1B parameters; treat that as the vendor’s claim (",{"tag":155,"href":197,"children":198},"https:\u002F\u002Fhuggingface.co\u002Fnaver\u002Fsplade-code-06B",[199],"SPLADE-Code",").",{"type":202,"variant":203,"title":204,"body":205},"callout","warn","Naver’s checkpoints are non-commercial",[206],[207,208,212],"The naver\u002Fsplade repository and Naver’s official checkpoints, including SPLADE-v3, are licensed CC BY-NC-SA 4.0. That rules them out for most commercial products. Apache-2.0 community re-hosts such as Qdrant\u002FSplade_PP_en_v1 exist (",{"tag":155,"href":209,"children":210},"https:\u002F\u002Fhuggingface.co\u002FQdrant\u002FSplade_PP_en_v1",[211],"Qdrant","), and Elastic’s ELSER and OpenSearch’s sparse models are vendor alternatives. Audit the license of the exact checkpoint before it reaches production.",{"type":214,"head":215,"rows":226},"table",[216,218,220,222,224],[217],"Model",[219],"Size",[221],"License",[223],"Where it runs",[225],"Notes",[227,237,245,256,265,275],[228,229,231,233,235],[168],[230],"BERT-base",[232],"CC BY-NC-SA 4.0",[234],"any engine with sparse vectors",[236],"40.2 MRR@10, 51.7 BEIR-13",[238,240,241,242,243],[239],"SPLADE++ (selfdistil)",[230],[232],[234],[244],"50.7 BEIR-13",[246,248,250,252,254],[247],"Qdrant\u002FSplade_PP_en_v1",[249],"BERT-base, ONNX",[251],"Apache-2.0",[253],"Qdrant, FastEmbed, others",[255],"commercial-friendly re-host",[257,259,261,262,263],[258],"efficient-SPLADE",[260],"separate query and doc encoders",[232],[234],[264],"within 4 ms of BM25 latency",[266,268,270,272,273],[267],"ELSER v2",[269],"BERT-base, about 100M",[271],"Elastic, with subscription",[14],[274],"English only, 512 tokens",[276,278,280,282,284],[277],"OpenSearch sparse models",[279],"various",[281],"see model card",[283],"OpenSearch",[285],"doc-only and bi-encoder modes",{"type":146,"level":147,"id":110,"text":111},{"type":129,"content":288},[289,290,200],"Expansion makes posting lists longer, so a learned sparse index is larger and slower to scan than a BM25 index over the same text; how much depends on the regularization strength of the model, and I found no single primary number for the ratio. The measured costs are concrete, though. OpenSearch’s benchmark on MS MARCO put BM25 at 5.4 ms fixed latency plus 2.80 ms per million documents, and doc-only neural sparse at 8.7 ms plus 5.15 ms per million, about 1.8 times BM25’s scan (",{"tag":155,"href":50,"children":291},[283],{"type":171,"src":293,"alt":294,"wide":174,"caption":295},"\u002Fimages\u002Fblog\u002Fsplade-learned-sparse-retrieval\u002Fdiagram-18.webp","Index scan cost per million documents Three horizontal bars measured in OpenSearch 2.12 on MS MARCO. BM25 scans the inverted index at 2.80 ms per million documents with 5.4 ms of fixed latency. Doc-only neural sparse scans at 5.15 ms per million documents with 8.7 ms fixed, about 1.8 times BM25. Bi-encoder neural sparse, which runs a model per query, scans at 20.58 ms per million documents with 265.4 ms fixed.","Doc-only learned sparse costs about twice BM25 per million documents; running a model per query is the expensive part.",{"type":129,"content":297},[298,299,200],"The expensive mode is running a model on every query. In the same benchmark, bi-encoder mode added 265.4 ms of fixed latency and 20.58 ms per million documents; OpenSearch’s two-phase processor, which first scores with high-weight tokens and then refines, cut average latency from 53.56 to 38.61 ms in doc-only mode and from 300.79 to 121.64 ms in bi-encoder mode. Naver’s efficiency work reached latency within 4 ms of BM25 with less than 10 percent MRR@10 loss using separate, lighter query encoders (",{"tag":155,"href":38,"children":300},[184],{"type":129,"content":302},[303,304,307,308,311,312,200],"Pruning is the cheapest lever. Static pruning of learned sparse indexes gave a 2x speedup for at most 2 percent effectiveness loss, or 4x for at most 8 percent, and rerankers stayed robust on candidates from pruned indexes (",{"tag":155,"href":47,"children":305},[306],"static pruning","). OpenSearch’s ingest processor supports several prune types, and ",{"tag":187,"children":309},[310],"max_ratio"," at 0.1 cuts storage by about 40 percent for less than 1 percent relevance loss (",{"tag":155,"href":53,"children":313},[283],{"type":146,"level":147,"id":113,"text":114},{"type":129,"content":316},[317,318,321,322,325,326,329,330,200],"Both engines store learned sparse vectors as weighted terms on the inverted index. In Elasticsearch, ",{"tag":187,"children":319},[320],"sparse_vector"," is the field type to use; token pruning is on by default since 9.1, sparse vectors are excluded from ",{"tag":187,"children":323},[324],"_source"," by default since 9.2, and weights are stored with 9 significant bits, about 0.4 percent relative error (",{"tag":155,"href":56,"children":327},[328],"Elastic","). Elastic’s own model, ELSER v2, is English-only, encodes the first 512 tokens of a field and won 10 of 12 BEIR datasets against BM25 with an average gain of 18 percent nDCG@10 (",{"tag":155,"href":59,"children":331},[20],{"type":129,"content":333},[334,335,338,339,342,343,347,348,350],"ELSER’s query-side optimisation is worth copying conceptually: prune query tokens that are much more frequent than expected, keep tokens above 0.4 of the maximum weight, and rescore the top 50 with the full token set. Elastic reports an average 3.4x speedup for a negligible change in nDCG@10 (",{"tag":155,"href":62,"children":336},[337],"Elastic Search Labs","). For fusion, Elasticsearch’s RRF retriever combines a BM25 standard retriever with an ELSER retriever using a default ",{"tag":187,"children":340},[341],"rank_constant"," of 60 (",{"tag":155,"href":344,"children":345},"https:\u002F\u002Fwww.elastic.co\u002Fdocs\u002Freference\u002Felasticsearch\u002Frest-apis\u002Freciprocal-rank-fusion",[346],"RRF","). OpenSearch offers doc-only and bi-encoder neural sparse modes, a two-phase processor, and since 3.3 approximate search on ",{"tag":187,"children":349},[320]," fields.",{"type":129,"content":352},[353,354,358,359,363],"Dedicated vector stores support sparse vectors too, which matters if you already run one; the ",{"tag":140,"to":355,"children":356},"\u002Fblog\u002Fqdrant",[357],"Qdrant review"," covers that side. The general mechanics of running vectors inside a search cluster are in ",{"tag":140,"to":360,"children":361},"\u002Fblog\u002Felasticsearch-opensearch-vector-search",[362],"vector search in Elasticsearch and OpenSearch",".",{"type":129,"content":365},[366],"In practice the choice of engine follows the cluster you already run. If you are on Elasticsearch with an Elastic subscription and English content, ELSER is the least work. If you are on OpenSearch, its own sparse models and the doc-only mode are the natural start. If you want an open SPLADE checkpoint in either engine, encode documents yourself in an ingest job and write the token weights into a sparse field, which both engines accept.",{"type":146,"level":147,"id":116,"text":117},{"type":129,"content":369},[370],"Learned sparse retrieval helps most when queries and documents use different words for the same thing: user questions against technical documentation, natural-language product searches against terse catalog text, synonyms and paraphrases in English. It also gives you a second retriever whose results you can explain term by term, which dense vectors cannot offer. In B2B catalogs from Spryker and Pimcore work, full of article numbers and normalized attributes, that combination of expansion and exact terms is the main motivation.",{"type":129,"content":372},[373,374,363],"It helps less when queries are mostly exact identifiers that BM25 already matches, when the content is not English and the model is English-only, as ELSER is, or when the corpus is so large that a second, longer posting list is the expensive part. It is not uniformly better on benchmarks either; the SPLADE++ results show BM25 still winning on some BEIR sets. Product search with part numbers specifically is covered in ",{"tag":140,"to":375,"children":376},"\u002Fblog\u002Fsemantic-product-search-b2b",[377],"semantic product search for B2B",{"type":146,"level":147,"id":119,"text":120},{"type":129,"content":380},[381,382,384,385,200],"Adapting a sparse model is within reach of one GPU. The naver\u002Fsplade repository ships configurations per training scenario, including a single-GPU configuration on a 16 GB card that reaches 37.2 MRR@10 (",{"tag":155,"href":38,"children":383},[184],"). Sentence Transformers now has a first-class SparseEncoder with SPLADE-specific losses such as SpladeLoss, CachedSpladeLoss and FlopsLoss, and supports asymmetric query and document encoding (",{"tag":155,"href":65,"children":386},[387],"Sentence Transformers",{"type":129,"content":389},[390],"For languages other than English, adaptation is often the only path, because most published sparse checkpoints, ELSER among them, are English-only. A multilingual base model with a BERT-style vocabulary can be trained into a sparse encoder with the same losses, but the vocabulary itself limits what can be expanded; German compounds, for example, are split into word pieces, and the expansion works on those pieces rather than on whole words.",{"type":129,"content":392},[393,394,398],"The data side is the same as for dense fine-tuning: query-document pairs from logs or generated queries, mined hard negatives and a frozen evaluation set; ",{"tag":140,"to":395,"children":396},"\u002Fblog\u002Fembedding-fine-tuning-domain",[397],"fine-tuning embedding models"," covers that loop. The sparsity regularizer is the extra knob, and it is worth sweeping, because it trades quality against index size and latency directly.",{"type":146,"level":147,"id":122,"text":123},{"type":401,"ordered":174,"items":402},"list",[403,405,407,409,411,413,415],[404],"Measure BM25 and your current hybrid on a labelled query set first.",[406],"Check the license of the exact checkpoint; Naver’s are non-commercial.",[408],"Check the model’s language coverage against your content.",[410],"Start in doc-only mode and prune tokens before measuring latency.",[412],"Fuse with BM25 or dense results through RRF, then compare against fusion without it.",[414],"Measure index size and p95 latency on your real document count.",[416],"Keep it only if recall at your k improves enough to pay for the second index.",{"type":129,"content":418},[419,420,363],"If you are deciding whether a learned sparse retriever earns its place next to BM25 and dense search, see my ",{"tag":140,"to":421,"children":422},"\u002Fexpertise\u002Fai-engineer",[423],"AI engineering work",{"type":146,"level":147,"id":125,"text":126},{"type":401,"ordered":174,"items":426},[427,430,433,436,439,442,445,448,451,454],[428],{"tag":155,"href":38,"children":429},[37],[431],{"tag":155,"href":41,"children":432},[40],[434],{"tag":155,"href":44,"children":435},[43],[437],{"tag":155,"href":47,"children":438},[46],[440],{"tag":155,"href":50,"children":441},[49],[443],{"tag":155,"href":53,"children":444},[52],[446],{"tag":155,"href":56,"children":447},[55],[449],{"tag":155,"href":59,"children":450},[58],[452],{"tag":155,"href":62,"children":453},[61],[455],{"tag":155,"href":65,"children":456},[64],[458,534,592,650],{"slug":459,"published":460,"minutes":461,"category":8,"tags":462,"keywords":468,"about":481,"sources":491,"cover":528,"og":529,"expertise":68,"locales":530,"lang":70,"title":531,"description":532,"coverAlt":533},"embedding-model-selection-2026","2026-10-17",9,[463,464,465,466,467],"Embeddings","RAG","MTEB","Vector search","Model selection",[469,470,471,472,473,474,475,476,477,478,479,480],"best embedding model 2026","how to choose an embedding model","embedding model comparison","MTEB leaderboard explained","RTEB benchmark","open source embedding model","multilingual embedding model","voyage-4 vs openai embeddings","Qwen3-Embedding","EmbeddingGemma","embedding model for RAG","switching embedding models re-embed",[482,485,488],{"name":483,"url":484},"Word embedding","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FWord_embedding",{"name":486,"url":487},"Information retrieval","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FInformation_retrieval",{"name":489,"url":490},"Discounted cumulative gain","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FDiscounted_cumulative_gain",[492,495,498,501,504,507,510,513,516,519,522,525],{"title":493,"url":494},"Enevoldsen et al.: MMTEB, Massive Multilingual Text Embedding Benchmark (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.13595",{"title":496,"url":497},"Hugging Face: Introducing RTEB, a new standard for retrieval evaluation (Oct 2025)","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Frteb",{"title":499,"url":500},"OpenAI: Vector embeddings guide","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fembeddings",{"title":502,"url":503},"OpenAI: API pricing","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fpricing",{"title":505,"url":506},"Voyage AI: The Voyage 4 model family, shared embedding space with MoE architecture (Jan 2026)","https:\u002F\u002Fblog.voyageai.com\u002F2026\u002F01\u002F15\u002Fvoyage-4\u002F",{"title":508,"url":509},"Voyage AI: Text embeddings documentation","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fembeddings",{"title":511,"url":512},"Voyage AI: Pricing","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fpricing",{"title":514,"url":515},"Google: Gemini API embeddings documentation","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fembeddings",{"title":517,"url":518},"Google: Gemini API pricing","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fpricing",{"title":520,"url":521},"Cohere: Embed models documentation","https:\u002F\u002Fdocs.cohere.com\u002Fdocs\u002Fcohere-embed",{"title":523,"url":524},"Qwen: Qwen3-Embedding-8B model card","https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-Embedding-8B",{"title":526,"url":527},"Google Developers Blog: Introducing EmbeddingGemma (Sep 2025)","https:\u002F\u002Fdevelopers.googleblog.com\u002Fen\u002Fintroducing-embeddinggemma\u002F","\u002Fimages\u002Fblog\u002Fembedding-model-selection-2026\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fembedding-model-selection-2026\u002Fog.jpg",[70,71,72],"Choosing an embedding model in 2026: leaderboards, costs and a bake-off on your own data","How to pick an embedding model for RAG in 2026: what MTEB and RTEB tell you, hosted vs open weights, prices, dimensions, and a one-week bake-off.","Three columns of stacked blocks, the middle tallest, each measured by a short bar and a wide bracket.",{"slug":535,"published":460,"minutes":536,"category":8,"tags":537,"keywords":541,"about":553,"sources":561,"cover":586,"og":587,"expertise":68,"locales":588,"lang":70,"title":589,"description":590,"coverAlt":591},"contextual-retrieval-late-chunking",10,[464,538,463,539,540],"Chunking","Contextual retrieval","Late chunking",[542,543,544,545,546,547,548,549,550,551,552],"contextual retrieval","anthropic contextual retrieval","late chunking","contextual chunk embeddings","contextual embeddings rag","contextual bm25","chunk context rag","voyage-context-3","voyage-context-4","late chunking vs contextual retrieval","rag chunking context loss",[554,557,558],{"name":555,"url":556},"Retrieval-augmented generation","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FRetrieval-augmented_generation",{"name":27,"url":28},{"name":559,"url":560},"Sentence embedding","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FSentence_embedding",[562,565,568,571,574,577,580,583],{"title":563,"url":564},"Anthropic: Introducing Contextual Retrieval (Sep 2024)","https:\u002F\u002Fwww.anthropic.com\u002Fnews\u002Fcontextual-retrieval",{"title":566,"url":567},"Günther et al.: Late Chunking, Contextual Chunk Embeddings Using Long-Context Embedding Models (2024, rev. 2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2409.04701",{"title":569,"url":570},"Jina AI: Late Chunking in Long-Context Embedding Models (Aug 2024)","https:\u002F\u002Fjina.ai\u002Fnews\u002Flate-chunking-in-long-context-embedding-models\u002F",{"title":572,"url":573},"Voyage AI: Introducing voyage-context-3 (Jul 2025)","https:\u002F\u002Fblog.voyageai.com\u002F2025\u002F07\u002F23\u002Fvoyage-context-3\u002F",{"title":575,"url":576},"Voyage AI: voyage-context-4 (Jun 2026)","https:\u002F\u002Fblog.voyageai.com\u002F2026\u002F06\u002F29\u002Fvoyage-context-4\u002F",{"title":578,"url":579},"Voyage AI: Contextualized chunk embeddings documentation","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fcontextualized-chunk-embeddings",{"title":581,"url":582},"Anthropic: Claude API pricing","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fabout-claude\u002Fpricing",{"title":584,"url":585},"Qu, Tu and Bao: Is Semantic Chunking Worth the Computational Cost? (2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.13070","\u002Fimages\u002Fblog\u002Fcontextual-retrieval-late-chunking\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fcontextual-retrieval-late-chunking\u002Fog.jpg",[70,71,72],"Contextual retrieval and late chunking: giving RAG chunks back their context","Chunks lose their document. How contextual retrieval, late chunking and contextualized embedding models fix that, what each costs, and when to skip it.","Four loose square pieces gathered by a wide bracket below, each joined upward to a large sheet.",{"slug":593,"published":460,"minutes":461,"category":8,"tags":594,"keywords":598,"about":611,"sources":619,"cover":644,"og":645,"expertise":68,"locales":646,"lang":70,"title":647,"description":648,"coverAlt":649},"rag-access-control-permissions",[464,595,596,597,466],"Access control","Security","Multi-tenancy",[599,600,601,602,603,604,605,606,607,608,609,610],"rag access control","permission aware rag","document level security rag","rag multi tenancy","vector database access control","pgvector row level security","security trimming azure ai search","openfga rag authorization","qdrant multitenancy","pinecone namespaces multitenancy","owasp llm08 vector and embedding weaknesses","rag data leakage",[612,615,618],{"name":613,"url":614},"Access-control list","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FAccess-control_list",{"name":616,"url":617},"Multitenancy","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FMultitenancy",{"name":555,"url":556},[620,623,626,629,632,635,638,641],{"title":621,"url":622},"OWASP GenAI Security Project: LLM08:2025 Vector and Embedding Weaknesses","https:\u002F\u002Fgenai.owasp.org\u002Fllmrisk\u002Fllm082025-vector-and-embedding-weaknesses\u002F",{"title":624,"url":625},"PostgreSQL documentation: Row Security Policies","https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002Fcurrent\u002Fddl-rowsecurity.html",{"title":627,"url":628},"pgvector README: filtering and iterative index scans","https:\u002F\u002Fgithub.com\u002Fpgvector\u002Fpgvector",{"title":630,"url":631},"Qdrant: Multitenancy and partitions guide","https:\u002F\u002Fqdrant.tech\u002Fdocumentation\u002Fguides\u002Fmultiple-partitions\u002F",{"title":633,"url":634},"Pinecone: Implement multitenancy using namespaces","https:\u002F\u002Fdocs.pinecone.io\u002Fguides\u002Findex-data\u002Fimplement-multitenancy",{"title":636,"url":637},"Microsoft Learn: Document-level access control in Azure AI Search","https:\u002F\u002Flearn.microsoft.com\u002Fen-us\u002Fazure\u002Fsearch\u002Fsearch-document-level-access-overview",{"title":639,"url":640},"OpenFGA: RAG authorization use case","https:\u002F\u002Fopenfga.dev\u002Fdocs\u002Fuse-cases\u002Frag-authorization",{"title":642,"url":643},"Morris et al.: Text Embeddings Reveal (Almost) As Much As Text (EMNLP 2023)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2310.06816","\u002Fimages\u002Fblog\u002Frag-access-control-permissions\u002Fcover.webp","\u002Fimages\u002Fblog\u002Frag-access-control-permissions\u002Fog.jpg",[70,71,72],"Access control for RAG: permission-aware retrieval without leaking documents","How RAG assistants leak documents and how to stop it: ACLs on every chunk, pre-filtering, row-level security in pgvector, permission sync and leak tests.","Seven cards in a row, six with a lock badge, one bare card sending an arrow outward.",{"slug":651,"published":460,"minutes":536,"category":8,"tags":652,"keywords":657,"about":669,"sources":677,"cover":704,"og":705,"expertise":706,"locales":707,"lang":70,"title":708,"description":709,"coverAlt":710},"rag-product-catalog-pim-data",[464,653,654,655,656],"PIM","Product data","B2B e-commerce","Metadata filtering",[658,659,660,661,662,663,664,665,666,667,668],"rag product catalog","rag pim data","llm product data","product catalog chatbot","akeneo rag","pimcore ai search","rag structured data","metadata filtering rag","product attributes embeddings","b2b product assistant","etim classification ai",[670,673,674],{"name":671,"url":672},"Product information management","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FProduct_information_management",{"name":555,"url":556},{"name":675,"url":676},"ETIM (standard)","https:\u002F\u002Fwww.etim-international.com\u002Fclassification\u002F",[678,681,684,687,689,692,695,698,701],{"title":679,"url":680},"Akeneo API: Products, product values, metric values and product models","https:\u002F\u002Fapi.akeneo.com\u002Fconcepts\u002Fproducts.html",{"title":682,"url":683},"Akeneo Help Center: What about products with variants?","https:\u002F\u002Fhelp.akeneo.com\u002Fserenity-take-the-power-over-your-products\u002Fserenity-what-about-products-with-variants",{"title":685,"url":686},"Pimcore documentation: Classification Store","https:\u002F\u002Fdocs.pimcore.com\u002Fplatform\u002FPimcore\u002FObjects\u002FObject_Classes\u002FData_Types\u002FClassification_Store\u002F",{"title":688,"url":676},"ETIM International: Classification",{"title":690,"url":691},"ETIM International: ETIM releases and the IXF format","https:\u002F\u002Fetim-international.com\u002F?p=754",{"title":693,"url":694},"Reddy et al.: Shopping Queries Dataset, a large-scale ESCI benchmark for improving product search (2022)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2206.06588",{"title":696,"url":697},"Amazon Science: esci-data repository","https:\u002F\u002Fgithub.com\u002Famazon-science\u002Fesci-data",{"title":699,"url":700},"Qdrant documentation: Filtering","https:\u002F\u002Fqdrant.tech\u002Fdocumentation\u002Fconcepts\u002Ffiltering\u002F",{"title":702,"url":703},"Sui et al.: Table Meets LLM, can large language models understand structured table data? (WSDM 2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.13062","\u002Fimages\u002Fblog\u002Frag-product-catalog-pim-data\u002Fcover.webp","\u002Fimages\u002Fblog\u002Frag-product-catalog-pim-data\u002Fog.jpg","b2b-ecommerce-developer",[70,71,72],"RAG over product catalogues: turning PIM data into answers that hold","Product data is structured, so chunking it like PDFs fails. How to serialize PIM attributes, route spec questions to filters and keep answers exact.","A frame of nine squares with tiny symbols, funnelling down to one card.",1792342719202]