[{"data":1,"prerenderedAt":775},["ShallowReactive",2],{"blog-elasticsearch-opensearch-vector-search-en":3},{"slug":4,"published":5,"updated":6,"minutes":7,"category":8,"tags":9,"keywords":15,"about":26,"sources":34,"cover":71,"og":72,"expertise":73,"locales":74,"lang":75,"title":78,"description":79,"coverAlt":80,"metaTitle":81,"takeaways":82,"faq":88,"toc":104,"blocks":132,"others":520},"elasticsearch-opensearch-vector-search","2026-09-30","2026-10-01",8,"rag",[10,11,12,13,14],"Elasticsearch","OpenSearch","Vector search","Hybrid search","Licensing",[16,17,18,19,20,21,22,23,24,25],"vector search in Elasticsearch and OpenSearch","elasticsearch knn vector search","opensearch neural search","hybrid search rrf","elasticsearch bbq better binary quantization","opensearch diskann jvector","elasticsearch agpl license","filtered knn search","elasticsearch vs opensearch vector database","search engine as vector store",[27,29,31],{"name":10,"url":28},"https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FElasticsearch",{"name":11,"url":30},"https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FOpenSearch_%28software%29",{"name":32,"url":33},"Nearest neighbor search","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FNearest_neighbor_search",[35,38,41,44,47,50,53,56,59,62,65,68],{"title":36,"url":37},"Elastic docs: kNN search","https:\u002F\u002Fwww.elastic.co\u002Fdocs\u002Fsolutions\u002Fsearch\u002Fvector\u002Fknn",{"title":39,"url":40},"Elastic docs: filtered kNN search","https:\u002F\u002Fwww.elastic.co\u002Fdocs\u002Fsolutions\u002Fsearch\u002Fvector\u002Fknn\u002Ffiltered-knn-search",{"title":42,"url":43},"Elastic docs: dense_vector field type","https:\u002F\u002Fwww.elastic.co\u002Fdocs\u002Freference\u002Felasticsearch\u002Fmapping-reference\u002Fdense-vector",{"title":45,"url":46},"Elastic docs: BBQ","https:\u002F\u002Fwww.elastic.co\u002Fdocs\u002Freference\u002Felasticsearch\u002Fmapping-reference\u002Fbbq",{"title":48,"url":49},"Elastic docs: reciprocal rank fusion","https:\u002F\u002Fwww.elastic.co\u002Fdocs\u002Freference\u002Felasticsearch\u002Frest-apis\u002Freciprocal-rank-fusion",{"title":51,"url":52},"Elastic docs: GPU vector indexing","https:\u002F\u002Fwww.elastic.co\u002Fdocs\u002Fsolutions\u002Fsearch\u002Fvector\u002Fgpu-vector-indexing",{"title":54,"url":55},"Elastic blog: Elasticsearch is open source again (Aug 2024)","https:\u002F\u002Fwww.elastic.co\u002Fblog\u002Felasticsearch-is-open-source-again",{"title":57,"url":58},"OpenSearch docs: methods and engines","https:\u002F\u002Fdocs.opensearch.org\u002Flatest\u002Fmappings\u002Fsupported-field-types\u002Fknn-methods-engines\u002F",{"title":60,"url":61},"OpenSearch docs: memory-optimized vectors","https:\u002F\u002Fdocs.opensearch.org\u002Flatest\u002Fmappings\u002Fsupported-field-types\u002Fknn-memory-optimized\u002F",{"title":63,"url":64},"OpenSearch docs: reciprocal rank fusion","https:\u002F\u002Fdocs.opensearch.org\u002Flatest\u002Fvector-search\u002Fai-search\u002Fhybrid-search\u002Frrf\u002F",{"title":66,"url":67},"OpenSearch docs: version history","https:\u002F\u002Fdocs.opensearch.org\u002Flatest\u002Fversion-history\u002F",{"title":69,"url":70},"AWS Open Source blog: introducing OpenSearch (Apr 2021)","https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fopensource\u002Fintroducing-opensearch\u002F","\u002Fimages\u002Fblog\u002Felasticsearch-opensearch-vector-search\u002Fcover.webp","\u002Fimages\u002Fblog\u002Felasticsearch-opensearch-vector-search\u002Fog.jpg","ai-engineer",[75,76,77],"en","de","hu","Vector search in Elasticsearch and OpenSearch: kNN, hybrid search and the license story","Elasticsearch and OpenSearch as vector stores: kNN and filters, BBQ and binary quantization, RRF hybrid search, GPU builds and the AGPL and Apache-2.0 split.","A cabinet with a text drawer and a vector drawer, searched by one magnifying glass.","Vector search in Elasticsearch, OpenSearch · Balázs Csorba",[83,84,85,86,87],"If you already run Elasticsearch or OpenSearch for product or site search, its built-in kNN is usually the first vector store to try, because hybrid search and filters come with it.","In Elasticsearch, a filter inside the kNN clause is applied during the graph search and returns k matches; a filter next to it runs afterwards and can return fewer.","Quantization decides the memory bill: Elasticsearch defaults to BBQ for 384 or more dimensions since 9.1, and OpenSearch offers 32x on-disk compression and 1, 2 or 4-bit binary quantization.","Both engines fuse BM25 and vector results with RRF; OpenSearch’s own BEIR test found RRF 3.86 percent below its score-based normalization on average.","Licensing differs: OpenSearch is Apache-2.0, Elasticsearch offers AGPL, SSPL or ELv2 since August 2024, and several vector features such as DiskBBQ and GPU indexing need an Elastic Enterprise subscription.",[89,92,95,98,101],{"q":90,"a":91},"Can Elasticsearch be used as a vector database?","Yes. The dense_vector field supports approximate kNN with HNSW by default, exact kNN for small candidate sets, filters during the search, quantization and hybrid search with BM25 through RRF. For many teams that already run Elasticsearch, it removes the need for a separate vector database.",{"q":93,"a":94},"What is the difference between Elasticsearch and OpenSearch for vector search?","OpenSearch is an Apache-2.0 fork of Elasticsearch 7.10.2 from 2021. Both offer HNSW kNN, quantization and RRF, but the details differ: Elasticsearch has BBQ and DiskBBQ, while OpenSearch has several engines including Lucene, Faiss and the DiskANN-style JVector, plus on-disk compression modes.",{"q":96,"a":97},"What is BBQ in Elasticsearch?","Better Binary Quantization, introduced in Elasticsearch 8.16 in September 2024, compresses each vector dimension to a single bit for up to 32x compression, while raw vectors stay on disk for rescoring. Since 9.1 it is the default index type for vectors with 384 or more dimensions.",{"q":99,"a":100},"Is Elasticsearch open source again?","Since August 2024 Elastic offers the AGPL v3 as a third license option next to SSPL and the Elastic License 2.0. OpenSearch remains under Apache-2.0. Which one fits depends on how you distribute or host your product, so check with whoever handles licensing in your company.",{"q":102,"a":103},"When is a dedicated vector database better than Elasticsearch or OpenSearch?","When vectors are the main workload rather than an add-on to text search, when the index is too large to keep in page cache affordably, or when you need features like multi-tenant namespaces at very large scale. Then a dedicated store such as Qdrant, Milvus or a managed service can be simpler and cheaper.",[105,108,111,114,117,120,123,126,129],{"id":106,"title":107},"do-you-need-a-vector-database","Do you need a vector database if you already run a search cluster?",{"id":109,"title":110},"how-knn-works","How kNN and filtering work in both engines",{"id":112,"title":113},"quantization-and-memory","Quantization is where the memory goes",{"id":115,"title":116},"hybrid-search-with-rrf","Hybrid search with RRF",{"id":118,"title":119},"gpu-and-ingestion","GPU index builds and ingestion",{"id":121,"title":122},"licensing","The license story and what it means for you",{"id":124,"title":125},"when-not-to-use","When Elasticsearch or OpenSearch is the wrong vector store",{"id":127,"title":128},"checklist","A practical checklist",{"id":130,"title":131},"sources","Sources",[133,150,164,167,176,184,187,188,214,220,230,236,237,265,269,296,299,302,303,328,330,341,344,345,364,374,377,378,388,434,437,444,445,453,456,457,474,481,482],{"type":134,"content":135},"paragraph",[136,140,141,145,146,149],{"tag":137,"children":138},"strong",[139],"Vector search in Elasticsearch and OpenSearch"," means using the search engine you may already run as the vector store: a ",{"tag":142,"children":143},"code",[144],"dense_vector"," or ",{"tag":142,"children":147},[148],"knn_vector"," field next to your text fields, approximate nearest-neighbour search on it, and hybrid ranking with BM25 in the same request. For a shop or a site that already has a search cluster, that is often simpler than adding a separate vector database, as long as memory and licensing work out.",{"type":134,"content":151},[152,153,158,159,163],"This article covers how kNN and filtering work in both engines, where quantization moves the memory bill, hybrid fusion with RRF, GPU index builds, the 2021 to 2024 license history, and when a dedicated store is the better choice. Dedicated options are compared in ",{"tag":154,"to":155,"children":156},"link","\u002Fblog\u002Fpgvector-vs-vector-databases",[157],"pgvector versus vector databases"," and the ",{"tag":154,"to":160,"children":161},"\u002Fblog\u002Fmilvus-zilliz",[162],"Milvus and Zilliz review",". Versions are as of October 2026: Elasticsearch 9.5 and OpenSearch 3.9.",{"type":165,"level":166,"id":106,"text":107},"heading",2,{"type":134,"content":168},[169,170,175],"Often not. If your text search already lives in Elasticsearch or OpenSearch, adding a vector field gives you semantic retrieval, keyword retrieval, filters on structured fields and fusion in one query, with one system to operate and back up. The cost is memory: approximate kNN with HNSW performs well only when the vector data fits in the node’s page cache (",{"tag":171,"href":172,"children":173},"a","https:\u002F\u002Fwww.elastic.co\u002Fdocs\u002Fsolutions\u002Fsearch\u002Fvector\u002Fknn\u002Fapproximate-knn",[174],"Elastic","). That one constraint drives most of the decisions below.",{"type":134,"content":177},[178,179,183],"B2B product search is the typical case: queries mix part numbers, brand filters and loose descriptions, so keyword and vector retrieval need each other. That pattern is described in ",{"tag":154,"to":180,"children":181},"\u002Fblog\u002Fsemantic-product-search-b2b",[182],"semantic product search for B2B",". Many shops on Spryker or Pimcore stacks already run such a cluster for the catalog, which makes the engine’s own vector support the obvious first experiment.",{"type":134,"content":185},[186],"Operationally, one cluster also means one set of backups, one access model and one place where documents are updated. When a product changes, its text fields and its embedding are written in the same document update, so the keyword index and the vector index cannot drift apart. With a separate vector database you have to build and monitor that synchronisation yourself, including deletes, which is where most stale-result bugs come from.",{"type":165,"level":166,"id":109,"text":110},{"type":134,"content":189},[190,191,194,195,197,198,200,201,203,204,206,207,210,211,213],"Elasticsearch offers three entry points for approximate kNN, the top-level ",{"tag":142,"children":192},[193],"knn"," option, a ",{"tag":142,"children":196},[193]," query and a ",{"tag":142,"children":199},[193]," retriever, plus exact brute-force kNN for small candidate sets; approximate search is on by default for ",{"tag":142,"children":202},[144]," fields (",{"tag":171,"href":37,"children":205},[174],"). The setting that matters most is where the filter goes. A filter inside the kNN clause is applied during the approximate search, so you get k matching documents. A filter placed next to the kNN clause in a ",{"tag":142,"children":208},[209],"bool"," query runs afterwards and can return fewer than k (",{"tag":171,"href":40,"children":212},[174],").",{"type":215,"src":216,"alt":217,"wide":218,"caption":219},"image","\u002Fimages\u002Fblog\u002Felasticsearch-opensearch-vector-search\u002Fdiagram-8.webp","Query paths in Elasticsearch and OpenSearch Two lanes. Top lane, Elasticsearch: a query with a kNN clause and a filter inside it; the filter is applied while the HNSW graph is traversed; the result is k matching documents, which an RRF retriever fuses with a BM25 retriever. Bottom lane, OpenSearch: a hybrid query with a BM25 part and a knn_vector part; both run; a search pipeline normalizes the scores or applies the score-ranker RRF processor; the fused list is returned.",true,"Elasticsearch filters during the graph walk and fuses with retrievers; OpenSearch runs both halves of a hybrid query and fuses in a search pipeline.",{"type":134,"content":221},[222,223,226,227,229],"The second knob is ",{"tag":142,"children":224},[225],"num_candidates",": how many candidates each shard collects from the graph before the top k are chosen. Raising it improves recall and costs latency, and it is the first setting to tune when results look slightly off. Keep k close to what you actually show or pass on, for example 50 for a reranker or 10 for an answer, and raise ",{"tag":142,"children":228},[225]," to two to four times that while you measure recall against exact kNN on a sample.",{"type":134,"content":231},[232,233,235],"OpenSearch gives you a choice of engine per field: Lucene with HNSW and efficient native filtering, Faiss with HNSW and IVF plus scalar, product and binary encoders, the deprecated NMSLIB, and JVector, a pure-Java DiskANN-style engine with product quantization and concurrent inserts (",{"tag":171,"href":58,"children":234},[11],"). For most new indexes Lucene or Faiss is the starting point; JVector is the option when the index should live mostly on disk.",{"type":165,"level":166,"id":112,"text":113},{"type":134,"content":238},[239,240,243,244,247,248,250,251,254,255,257,258,261,262,213],"Both engines now quantize by default or with one setting, and the defaults changed quickly. In Elasticsearch, 9.0 defaulted to ",{"tag":142,"children":241},[242],"int8_hnsw","; 9.1 and later default to ",{"tag":142,"children":245},[246],"bbq_hnsw"," for vectors with 384 or more dimensions and ",{"tag":142,"children":249},[242]," otherwise; and 9.4 and later default to ",{"tag":142,"children":252},[253],"bbq_disk"," when the license allows (",{"tag":171,"href":43,"children":256},[174],"). BBQ arrived in 8.16 in September 2024 with up to 32 times compression (",{"tag":171,"href":259,"children":260},"https:\u002F\u002Fwww.elastic.co\u002Fblog\u002Fwhats-new-elastic-search-8-16-0",[174],"), and the docs put the memory reduction at 96 percent per dimension, with the raw vectors kept on disk for rescoring (",{"tag":171,"href":46,"children":263},[264],"BBQ",{"type":215,"src":266,"alt":267,"wide":218,"caption":268},"\u002Fimages\u002Fblog\u002Felasticsearch-opensearch-vector-search\u002Fdiagram-13.webp","Memory per vector by quantization level Four levels from left to right: raw float32 vectors at full size; int8 scalar quantization at about a quarter of the size, the Elasticsearch 9.0 default; int4 at about an eighth; and one bit per dimension with BBQ or binary quantization at up to 32 times smaller, the Elasticsearch default for 384 or more dimensions since 9.1 and the OpenSearch on-disk default for float vectors.","Each step trades memory for recall, which rescoring against the raw vectors on disk wins back.",{"type":134,"content":270},[271,272,274,275,277,278,281,282,285,286,289,290,292,293,213],"DiskBBQ, the ",{"tag":142,"children":273},[253]," type, clusters vectors with hierarchical k-means on disk, is aimed at recall up to about 95 percent, and requires an Enterprise subscription (",{"tag":171,"href":46,"children":276},[264],"). OpenSearch reaches similar ratios differently: its memory-optimized vectors default to 1x compression in ",{"tag":142,"children":279},[280],"in_memory"," mode and 32x for ",{"tag":142,"children":283},[284],"float"," in ",{"tag":142,"children":287},[288],"on_disk"," mode, with a rescoring oversample factor of 2.0 at 32x (",{"tag":171,"href":61,"children":291},[11],"), and Faiss binary quantization supports 1, 2 or 4 bits for 32, 16 or 8 times compression (",{"tag":171,"href":294,"children":295},"https:\u002F\u002Fdocs.opensearch.org\u002Flatest\u002Fvector-search\u002Foptimizing-storage\u002Fbinary-quantization\u002F",[11],{"type":134,"content":297},[298],"A rough sizing helps before choosing a level. A million 1,024-dimensional float32 vectors take about 4 GB raw, about 1 GB as int8 and about 128 MB at one bit per dimension, plus graph overhead. For a catalog of a few hundred thousand products, int8 often fits comfortably and keeps recall high; BBQ starts to pay off at millions of vectors or on small nodes, where it decides whether the index fits in memory at all.",{"type":134,"content":300},[301],"No neutral head-to-head benchmark of BBQ against OpenSearch quantization turned up in my research; what exists is published by the vendors themselves. Measure recall at your k on your own vectors before and after switching levels, and keep rescoring on.",{"type":165,"level":166,"id":115,"text":116},{"type":134,"content":304},[305,306,309,310,313,314,317,318,145,321,324,325,327],"Both engines fuse keyword and vector results, but in different places. Elasticsearch added reciprocal rank fusion in 8.8 and made it generally available with retrievers in 8.16; the ",{"tag":142,"children":307},[308],"rrf"," retriever needs at least two child retrievers, defaults ",{"tag":142,"children":311},[312],"rank_constant"," to 60, and does not support ",{"tag":142,"children":315},[316],"sort",", ",{"tag":142,"children":319},[320],"scroll",{"tag":142,"children":322},[323],"rescore"," (",{"tag":171,"href":49,"children":326},[174],"). OpenSearch fuses in a search pipeline: score-based normalization since 2.10 and RRF as the score-ranker processor since 2.19.",{"type":142,"code":329},"PUT products\n{\n  \"mappings\": {\n    \"properties\": {\n      \"title\":     { \"type\": \"text\" },\n      \"brand\":     { \"type\": \"keyword\" },\n      \"embedding\": { \"type\": \"dense_vector\", \"dims\": 1024, \"similarity\": \"cosine\",\n                     \"index_options\": { \"type\": \"bbq_hnsw\" } }\n    }\n  }\n}\n\nGET products\u002F_search\n{\n  \"retriever\": {\n    \"rrf\": {\n      \"retrievers\": [\n        { \"standard\": { \"query\": { \"match\": { \"title\": \"Kugelhahn DN50 Edelstahl\" } } } },\n        { \"knn\": { \"field\": \"embedding\", \"query_vector\": [\u002F* 1024 floats *\u002F], \"k\": 50,\n                   \"num_candidates\": 200,\n                   \"filter\": { \"term\": { \"brand\": \"acme\" } } } }\n      ],\n      \"rank_constant\": 60\n    }\n  }\n}",{"type":134,"content":331},[332,333,335,336,340],"OpenSearch published a useful comparison: on six BEIR datasets, RRF averaged 3.86 percent lower NDCG@10 than its score-based hybrid pipeline at comparable latency (",{"tag":171,"href":64,"children":334},[11],"). RRF is still the safer default when score scales differ wildly, because it ignores scores entirely. If you want a model to refine the fused list afterwards, see ",{"tag":154,"to":337,"children":338},"\u002Fblog\u002Freranking-models-compared",[339],"cross-encoder rerankers compared",".",{"type":134,"content":342},[343],"In practice the fusion step is where relevance problems show up first. A query that is just a part number should be won by BM25, while a vague description should be won by vectors, and RRF handles both without tuning. Log which retriever contributed each result for a sample of real queries; if one side never contributes, either its query is wrong or it is not worth its cost.",{"type":165,"level":166,"id":118,"text":119},{"type":134,"content":346},[347,348,351,352,354,355,357,358,360,361,213],"Building HNSW graphs is the expensive part of ingestion, and both projects now offload it to NVIDIA cuVS. In Elasticsearch, GPU vector indexing was a preview in 9.3 and generally available in 9.4; it requires an Enterprise subscription, Linux on x86_64, an Ampere or newer GPU with at least 8 GB, and works only for ",{"tag":142,"children":349},[350],"hnsw"," and ",{"tag":142,"children":353},[242]," with ",{"tag":142,"children":356},[284]," elements (",{"tag":171,"href":52,"children":359},[174],"). OpenSearch shipped GPU acceleration as a preview in 3.0 and production-ready in 3.1, reporting index builds up to 9.3 times faster and 3.75 times cheaper than CPU-only (",{"tag":171,"href":362,"children":363},"https:\u002F\u002Fopensearch.org\u002Fblog\u002Fgpu-accelerated-vector-search-opensearch-new-frontier\u002F",[11],{"type":134,"content":365},[366,367,370,371,373],"Smaller ingestion details add up as well. OpenSearch 3.8 reports four times faster vector ingestion with Base64-encoded vectors, and 3.9 added the ",{"tag":142,"children":368},[369],"half_float"," vector type and 2-bit and 4-bit scalar quantization (",{"tag":171,"href":67,"children":372},[11],"). Halving the float width before indexing is often the cheapest memory win available.",{"type":134,"content":375},[376],"For a catalog that changes daily rather than every second, GPU builds are rarely the bottleneck; the bigger ingestion cost is usually generating the embeddings themselves. Batch the embedding calls, skip documents whose embedded text has not changed, and re-embed everything only when you switch models. A GPU index build becomes interesting for full rebuilds of tens of millions of vectors, where it shortens the window in which a new index is being prepared.",{"type":165,"level":166,"id":121,"text":122},{"type":134,"content":379},[380,381,384,385,387],"In April 2021 AWS announced OpenSearch as a community fork of Elasticsearch and Kibana 7.10.2 under the Apache License 2.0 (",{"tag":171,"href":70,"children":382},[383],"AWS","), after Elastic had moved to SSPL and its own Elastic License. On 29 August 2024 Elastic added the AGPL v3 as a third option next to ELv2 and SSPL (",{"tag":171,"href":55,"children":386},[174],"). OpenSearch remains plain Apache-2.0.",{"type":389,"head":390,"rows":401},"table",[391,393,395,397,399],[392],"Situation",[394],"Vector setup",[396],"Hybrid fusion",[398],"GPU builds",[400],"License exposure",[402,413,424],[403,405,407,409,411],[404],"Already on Elasticsearch",[406],"dense_vector, bbq_hnsw default",[408],"rrf retriever",[410],"Enterprise only",[412],"AGPL, SSPL or ELv2; DiskBBQ needs Enterprise",[414,416,418,420,422],[415],"Already on OpenSearch",[417],"knn_vector, Lucene or Faiss",[419],"normalization or score-ranker",[421],"available since 3.1",[423],"Apache-2.0",[425,427,429,431,432],[426],"Greenfield, vectors are main workload",[428],"consider a dedicated store",[430],"depends on store",[430],[433],"check each store",{"type":134,"content":435},[436],"The practical consequence is in the feature list, not just the license text. Several of the Elasticsearch vector features covered above, DiskBBQ and GPU indexing among them, require an Enterprise subscription, while the matching OpenSearch features ship in the Apache-2.0 distribution. Managed offerings from cloud providers add their own terms on top, so compare the feature you actually need in the edition you would actually run.",{"type":438,"variant":439,"title":440,"body":441},"callout","note","Licensing is a product decision",[442],[443],"Running a search cluster behind your own application is different from offering it as a service or shipping it inside a product. The differences between AGPL, SSPL, ELv2 and Apache-2.0 matter exactly in those cases, so get the question answered by whoever handles licensing before you build on paid-tier features.",{"type":165,"level":166,"id":124,"text":125},{"type":134,"content":446},[447,448,452],"A dedicated vector store wins when vectors are the main workload rather than an add-on to text search, when the index is too large to keep in page cache at a price you accept, or when you need per-tenant namespaces, very high write rates or vector-native features that the search engines do not offer. It also wins when nobody on the team wants to operate a JVM search cluster in the first place. In those cases, start from the dedicated options in ",{"tag":154,"to":449,"children":450},"\u002Fblog\u002Fpgvector",[451],"pgvector"," or the comparison posts above.",{"type":134,"content":454},[455],"Moving later is not expensive if you plan for it. Keep the embedding model, its version and the text that was embedded next to each vector, and write documents through one indexing service rather than from several places. Then switching the vector store, or adding a dedicated one for a single workload, means re-running ingestion against a new target instead of rebuilding the whole pipeline.",{"type":165,"level":166,"id":127,"text":128},{"type":458,"ordered":218,"items":459},"list",[460,462,464,466,468,470,472],[461],"Estimate vector memory at your dimension count and quantization level, and compare it with node page cache.",[463],"Put filters inside the kNN clause, not next to it.",[465],"Start with the version’s default quantization and keep rescoring on.",[467],"Fuse BM25 and vectors with RRF first, then test score normalization.",[469],"Measure recall at k with exact kNN as ground truth on a sample.",[471],"Check which features need a paid subscription before designing around them.",[473],"Re-test after every minor upgrade, because defaults change between versions.",{"type":134,"content":475},[476,477,340],"If you want vector search added to a search cluster you already run, with recall measured rather than assumed, see my ",{"tag":154,"to":478,"children":479},"\u002Fexpertise\u002Fai-engineer",[480],"AI engineering work",{"type":165,"level":166,"id":130,"text":131},{"type":458,"ordered":218,"items":483},[484,487,490,493,496,499,502,505,508,511,514,517],[485],{"tag":171,"href":37,"children":486},[36],[488],{"tag":171,"href":40,"children":489},[39],[491],{"tag":171,"href":43,"children":492},[42],[494],{"tag":171,"href":46,"children":495},[45],[497],{"tag":171,"href":49,"children":498},[48],[500],{"tag":171,"href":52,"children":501},[51],[503],{"tag":171,"href":55,"children":504},[54],[506],{"tag":171,"href":58,"children":507},[57],[509],{"tag":171,"href":61,"children":510},[60],[512],{"tag":171,"href":64,"children":513},[63],[515],{"tag":171,"href":67,"children":516},[66],[518],{"tag":171,"href":70,"children":519},[69],[521,596,656,714],{"slug":522,"published":523,"minutes":524,"category":8,"tags":525,"keywords":530,"about":543,"sources":553,"cover":590,"og":591,"expertise":73,"locales":592,"lang":75,"title":593,"description":594,"coverAlt":595},"embedding-model-selection-2026","2026-10-17",9,[526,527,528,12,529],"Embeddings","RAG","MTEB","Model selection",[531,532,533,534,535,536,537,538,539,540,541,542],"best embedding model 2026","how to choose an embedding model","embedding model comparison","MTEB leaderboard explained","RTEB benchmark","open source embedding model","multilingual embedding model","voyage-4 vs openai embeddings","Qwen3-Embedding","EmbeddingGemma","embedding model for RAG","switching embedding models re-embed",[544,547,550],{"name":545,"url":546},"Word embedding","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FWord_embedding",{"name":548,"url":549},"Information retrieval","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FInformation_retrieval",{"name":551,"url":552},"Discounted cumulative gain","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FDiscounted_cumulative_gain",[554,557,560,563,566,569,572,575,578,581,584,587],{"title":555,"url":556},"Enevoldsen et al.: MMTEB, Massive Multilingual Text Embedding Benchmark (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.13595",{"title":558,"url":559},"Hugging Face: Introducing RTEB, a new standard for retrieval evaluation (Oct 2025)","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Frteb",{"title":561,"url":562},"OpenAI: Vector embeddings guide","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fembeddings",{"title":564,"url":565},"OpenAI: API pricing","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fpricing",{"title":567,"url":568},"Voyage AI: The Voyage 4 model family, shared embedding space with MoE architecture (Jan 2026)","https:\u002F\u002Fblog.voyageai.com\u002F2026\u002F01\u002F15\u002Fvoyage-4\u002F",{"title":570,"url":571},"Voyage AI: Text embeddings documentation","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fembeddings",{"title":573,"url":574},"Voyage AI: Pricing","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fpricing",{"title":576,"url":577},"Google: Gemini API embeddings documentation","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fembeddings",{"title":579,"url":580},"Google: Gemini API pricing","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fpricing",{"title":582,"url":583},"Cohere: Embed models documentation","https:\u002F\u002Fdocs.cohere.com\u002Fdocs\u002Fcohere-embed",{"title":585,"url":586},"Qwen: Qwen3-Embedding-8B model card","https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-Embedding-8B",{"title":588,"url":589},"Google Developers Blog: Introducing EmbeddingGemma (Sep 2025)","https:\u002F\u002Fdevelopers.googleblog.com\u002Fen\u002Fintroducing-embeddinggemma\u002F","\u002Fimages\u002Fblog\u002Fembedding-model-selection-2026\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fembedding-model-selection-2026\u002Fog.jpg",[75,76,77],"Choosing an embedding model in 2026: leaderboards, costs and a bake-off on your own data","How to pick an embedding model for RAG in 2026: what MTEB and RTEB tell you, hosted vs open weights, prices, dimensions, and a one-week bake-off.","Three columns of stacked blocks, the middle tallest, each measured by a short bar and a wide bracket.",{"slug":597,"published":523,"minutes":598,"category":8,"tags":599,"keywords":603,"about":615,"sources":625,"cover":650,"og":651,"expertise":73,"locales":652,"lang":75,"title":653,"description":654,"coverAlt":655},"contextual-retrieval-late-chunking",10,[527,600,526,601,602],"Chunking","Contextual retrieval","Late chunking",[604,605,606,607,608,609,610,611,612,613,614],"contextual retrieval","anthropic contextual retrieval","late chunking","contextual chunk embeddings","contextual embeddings rag","contextual bm25","chunk context rag","voyage-context-3","voyage-context-4","late chunking vs contextual retrieval","rag chunking context loss",[616,619,622],{"name":617,"url":618},"Retrieval-augmented generation","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FRetrieval-augmented_generation",{"name":620,"url":621},"Okapi BM25","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FOkapi_BM25",{"name":623,"url":624},"Sentence embedding","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FSentence_embedding",[626,629,632,635,638,641,644,647],{"title":627,"url":628},"Anthropic: Introducing Contextual Retrieval (Sep 2024)","https:\u002F\u002Fwww.anthropic.com\u002Fnews\u002Fcontextual-retrieval",{"title":630,"url":631},"Günther et al.: Late Chunking, Contextual Chunk Embeddings Using Long-Context Embedding Models (2024, rev. 2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2409.04701",{"title":633,"url":634},"Jina AI: Late Chunking in Long-Context Embedding Models (Aug 2024)","https:\u002F\u002Fjina.ai\u002Fnews\u002Flate-chunking-in-long-context-embedding-models\u002F",{"title":636,"url":637},"Voyage AI: Introducing voyage-context-3 (Jul 2025)","https:\u002F\u002Fblog.voyageai.com\u002F2025\u002F07\u002F23\u002Fvoyage-context-3\u002F",{"title":639,"url":640},"Voyage AI: voyage-context-4 (Jun 2026)","https:\u002F\u002Fblog.voyageai.com\u002F2026\u002F06\u002F29\u002Fvoyage-context-4\u002F",{"title":642,"url":643},"Voyage AI: Contextualized chunk embeddings documentation","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fcontextualized-chunk-embeddings",{"title":645,"url":646},"Anthropic: Claude API pricing","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fabout-claude\u002Fpricing",{"title":648,"url":649},"Qu, Tu and Bao: Is Semantic Chunking Worth the Computational Cost? (2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.13070","\u002Fimages\u002Fblog\u002Fcontextual-retrieval-late-chunking\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fcontextual-retrieval-late-chunking\u002Fog.jpg",[75,76,77],"Contextual retrieval and late chunking: giving RAG chunks back their context","Chunks lose their document. How contextual retrieval, late chunking and contextualized embedding models fix that, what each costs, and when to skip it.","Four loose square pieces gathered by a wide bracket below, each joined upward to a large sheet.",{"slug":657,"published":523,"minutes":524,"category":8,"tags":658,"keywords":662,"about":675,"sources":683,"cover":708,"og":709,"expertise":73,"locales":710,"lang":75,"title":711,"description":712,"coverAlt":713},"rag-access-control-permissions",[527,659,660,661,12],"Access control","Security","Multi-tenancy",[663,664,665,666,667,668,669,670,671,672,673,674],"rag access control","permission aware rag","document level security rag","rag multi tenancy","vector database access control","pgvector row level security","security trimming azure ai search","openfga rag authorization","qdrant multitenancy","pinecone namespaces multitenancy","owasp llm08 vector and embedding weaknesses","rag data leakage",[676,679,682],{"name":677,"url":678},"Access-control list","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FAccess-control_list",{"name":680,"url":681},"Multitenancy","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FMultitenancy",{"name":617,"url":618},[684,687,690,693,696,699,702,705],{"title":685,"url":686},"OWASP GenAI Security Project: LLM08:2025 Vector and Embedding Weaknesses","https:\u002F\u002Fgenai.owasp.org\u002Fllmrisk\u002Fllm082025-vector-and-embedding-weaknesses\u002F",{"title":688,"url":689},"PostgreSQL documentation: Row Security Policies","https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002Fcurrent\u002Fddl-rowsecurity.html",{"title":691,"url":692},"pgvector README: filtering and iterative index scans","https:\u002F\u002Fgithub.com\u002Fpgvector\u002Fpgvector",{"title":694,"url":695},"Qdrant: Multitenancy and partitions guide","https:\u002F\u002Fqdrant.tech\u002Fdocumentation\u002Fguides\u002Fmultiple-partitions\u002F",{"title":697,"url":698},"Pinecone: Implement multitenancy using namespaces","https:\u002F\u002Fdocs.pinecone.io\u002Fguides\u002Findex-data\u002Fimplement-multitenancy",{"title":700,"url":701},"Microsoft Learn: Document-level access control in Azure AI Search","https:\u002F\u002Flearn.microsoft.com\u002Fen-us\u002Fazure\u002Fsearch\u002Fsearch-document-level-access-overview",{"title":703,"url":704},"OpenFGA: RAG authorization use case","https:\u002F\u002Fopenfga.dev\u002Fdocs\u002Fuse-cases\u002Frag-authorization",{"title":706,"url":707},"Morris et al.: Text Embeddings Reveal (Almost) As Much As Text (EMNLP 2023)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2310.06816","\u002Fimages\u002Fblog\u002Frag-access-control-permissions\u002Fcover.webp","\u002Fimages\u002Fblog\u002Frag-access-control-permissions\u002Fog.jpg",[75,76,77],"Access control for RAG: permission-aware retrieval without leaking documents","How RAG assistants leak documents and how to stop it: ACLs on every chunk, pre-filtering, row-level security in pgvector, permission sync and leak tests.","Seven cards in a row, six with a lock badge, one bare card sending an arrow outward.",{"slug":715,"published":523,"minutes":598,"category":8,"tags":716,"keywords":721,"about":733,"sources":741,"cover":768,"og":769,"expertise":770,"locales":771,"lang":75,"title":772,"description":773,"coverAlt":774},"rag-product-catalog-pim-data",[527,717,718,719,720],"PIM","Product data","B2B e-commerce","Metadata filtering",[722,723,724,725,726,727,728,729,730,731,732],"rag product catalog","rag pim data","llm product data","product catalog chatbot","akeneo rag","pimcore ai search","rag structured data","metadata filtering rag","product attributes embeddings","b2b product assistant","etim classification ai",[734,737,738],{"name":735,"url":736},"Product information management","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FProduct_information_management",{"name":617,"url":618},{"name":739,"url":740},"ETIM (standard)","https:\u002F\u002Fwww.etim-international.com\u002Fclassification\u002F",[742,745,748,751,753,756,759,762,765],{"title":743,"url":744},"Akeneo API: Products, product values, metric values and product models","https:\u002F\u002Fapi.akeneo.com\u002Fconcepts\u002Fproducts.html",{"title":746,"url":747},"Akeneo Help Center: What about products with variants?","https:\u002F\u002Fhelp.akeneo.com\u002Fserenity-take-the-power-over-your-products\u002Fserenity-what-about-products-with-variants",{"title":749,"url":750},"Pimcore documentation: Classification Store","https:\u002F\u002Fdocs.pimcore.com\u002Fplatform\u002FPimcore\u002FObjects\u002FObject_Classes\u002FData_Types\u002FClassification_Store\u002F",{"title":752,"url":740},"ETIM International: Classification",{"title":754,"url":755},"ETIM International: ETIM releases and the IXF format","https:\u002F\u002Fetim-international.com\u002F?p=754",{"title":757,"url":758},"Reddy et al.: Shopping Queries Dataset, a large-scale ESCI benchmark for improving product search (2022)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2206.06588",{"title":760,"url":761},"Amazon Science: esci-data repository","https:\u002F\u002Fgithub.com\u002Famazon-science\u002Fesci-data",{"title":763,"url":764},"Qdrant documentation: Filtering","https:\u002F\u002Fqdrant.tech\u002Fdocumentation\u002Fconcepts\u002Ffiltering\u002F",{"title":766,"url":767},"Sui et al.: Table Meets LLM, can large language models understand structured table data? (WSDM 2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.13062","\u002Fimages\u002Fblog\u002Frag-product-catalog-pim-data\u002Fcover.webp","\u002Fimages\u002Fblog\u002Frag-product-catalog-pim-data\u002Fog.jpg","b2b-ecommerce-developer",[75,76,77],"RAG over product catalogues: turning PIM data into answers that hold","Product data is structured, so chunking it like PDFs fails. How to serialize PIM attributes, route spec questions to filters and keep answers exact.","A frame of nine squares with tiny symbols, funnelling down to one card.",1792342719114]