Vector search in Elasticsearch and OpenSearch: kNN, hybrid search and the license story

Vector search in Elasticsearch and OpenSearch means using the search engine you may already run as the vector store: a dense_vector or knn_vector field next to your text fields, approximate nearest-neighbour search on it, and hybrid ranking with BM25 in the same request. For a shop or a site that already has a search cluster, that is often simpler than adding a separate vector database, as long as memory and licensing work out.

This article covers how kNN and filtering work in both engines, where quantization moves the memory bill, hybrid fusion with RRF, GPU index builds, the 2021 to 2024 license history, and when a dedicated store is the better choice. Dedicated options are compared in pgvector versus vector databases and the Milvus and Zilliz review. Versions are as of October 2026: Elasticsearch 9.5 and OpenSearch 3.9.

Do you need a vector database if you already run a search cluster?

Often not. If your text search already lives in Elasticsearch or OpenSearch, adding a vector field gives you semantic retrieval, keyword retrieval, filters on structured fields and fusion in one query, with one system to operate and back up. The cost is memory: approximate kNN with HNSW performs well only when the vector data fits in the node’s page cache (Elastic). That one constraint drives most of the decisions below.

B2B product search is the typical case: queries mix part numbers, brand filters and loose descriptions, so keyword and vector retrieval need each other. That pattern is described in semantic product search for B2B. Many shops on Spryker or Pimcore stacks already run such a cluster for the catalog, which makes the engine’s own vector support the obvious first experiment.

Operationally, one cluster also means one set of backups, one access model and one place where documents are updated. When a product changes, its text fields and its embedding are written in the same document update, so the keyword index and the vector index cannot drift apart. With a separate vector database you have to build and monitor that synchronisation yourself, including deletes, which is where most stale-result bugs come from.

How kNN and filtering work in both engines

Elasticsearch offers three entry points for approximate kNN, the top-level knn option, a knn query and a knn retriever, plus exact brute-force kNN for small candidate sets; approximate search is on by default for dense_vector fields (Elastic). The setting that matters most is where the filter goes. A filter inside the kNN clause is applied during the approximate search, so you get k matching documents. A filter placed next to the kNN clause in a bool query runs afterwards and can return fewer than k (Elastic).

The second knob is num_candidates: how many candidates each shard collects from the graph before the top k are chosen. Raising it improves recall and costs latency, and it is the first setting to tune when results look slightly off. Keep k close to what you actually show or pass on, for example 50 for a reranker or 10 for an answer, and raise num_candidates to two to four times that while you measure recall against exact kNN on a sample.

OpenSearch gives you a choice of engine per field: Lucene with HNSW and efficient native filtering, Faiss with HNSW and IVF plus scalar, product and binary encoders, the deprecated NMSLIB, and JVector, a pure-Java DiskANN-style engine with product quantization and concurrent inserts (OpenSearch). For most new indexes Lucene or Faiss is the starting point; JVector is the option when the index should live mostly on disk.

Quantization is where the memory goes

Both engines now quantize by default or with one setting, and the defaults changed quickly. In Elasticsearch, 9.0 defaulted to int8_hnsw; 9.1 and later default to bbq_hnsw for vectors with 384 or more dimensions and int8_hnsw otherwise; and 9.4 and later default to bbq_disk when the license allows (Elastic). BBQ arrived in 8.16 in September 2024 with up to 32 times compression (Elastic), and the docs put the memory reduction at 96 percent per dimension, with the raw vectors kept on disk for rescoring (BBQ).

DiskBBQ, the bbq_disk type, clusters vectors with hierarchical k-means on disk, is aimed at recall up to about 95 percent, and requires an Enterprise subscription (BBQ). OpenSearch reaches similar ratios differently: its memory-optimized vectors default to 1x compression in in_memory mode and 32x for float in on_disk mode, with a rescoring oversample factor of 2.0 at 32x (OpenSearch), and Faiss binary quantization supports 1, 2 or 4 bits for 32, 16 or 8 times compression (OpenSearch).

A rough sizing helps before choosing a level. A million 1,024-dimensional float32 vectors take about 4 GB raw, about 1 GB as int8 and about 128 MB at one bit per dimension, plus graph overhead. For a catalog of a few hundred thousand products, int8 often fits comfortably and keeps recall high; BBQ starts to pay off at millions of vectors or on small nodes, where it decides whether the index fits in memory at all.

No neutral head-to-head benchmark of BBQ against OpenSearch quantization turned up in my research; what exists is published by the vendors themselves. Measure recall at your k on your own vectors before and after switching levels, and keep rescoring on.

Hybrid search with RRF

Both engines fuse keyword and vector results, but in different places. Elasticsearch added reciprocal rank fusion in 8.8 and made it generally available with retrievers in 8.16; the rrf retriever needs at least two child retrievers, defaults rank_constant to 60, and does not support sort, scroll or rescore (Elastic). OpenSearch fuses in a search pipeline: score-based normalization since 2.10 and RRF as the score-ranker processor since 2.19.

PUT products
{
  "mappings": {
    "properties": {
      "title":     { "type": "text" },
      "brand":     { "type": "keyword" },
      "embedding": { "type": "dense_vector", "dims": 1024, "similarity": "cosine",
                     "index_options": { "type": "bbq_hnsw" } }
    }
  }
}

GET products/_search
{
  "retriever": {
    "rrf": {
      "retrievers": [
        { "standard": { "query": { "match": { "title": "Kugelhahn DN50 Edelstahl" } } } },
        { "knn": { "field": "embedding", "query_vector": [/* 1024 floats */], "k": 50,
                   "num_candidates": 200,
                   "filter": { "term": { "brand": "acme" } } } }
      ],
      "rank_constant": 60
    }
  }
}

OpenSearch published a useful comparison: on six BEIR datasets, RRF averaged 3.86 percent lower NDCG@10 than its score-based hybrid pipeline at comparable latency (OpenSearch). RRF is still the safer default when score scales differ wildly, because it ignores scores entirely. If you want a model to refine the fused list afterwards, see cross-encoder rerankers compared.

In practice the fusion step is where relevance problems show up first. A query that is just a part number should be won by BM25, while a vague description should be won by vectors, and RRF handles both without tuning. Log which retriever contributed each result for a sample of real queries; if one side never contributes, either its query is wrong or it is not worth its cost.

GPU index builds and ingestion

Building HNSW graphs is the expensive part of ingestion, and both projects now offload it to NVIDIA cuVS. In Elasticsearch, GPU vector indexing was a preview in 9.3 and generally available in 9.4; it requires an Enterprise subscription, Linux on x86_64, an Ampere or newer GPU with at least 8 GB, and works only for hnsw and int8_hnsw with float elements (Elastic). OpenSearch shipped GPU acceleration as a preview in 3.0 and production-ready in 3.1, reporting index builds up to 9.3 times faster and 3.75 times cheaper than CPU-only (OpenSearch).

Smaller ingestion details add up as well. OpenSearch 3.8 reports four times faster vector ingestion with Base64-encoded vectors, and 3.9 added the half_float vector type and 2-bit and 4-bit scalar quantization (OpenSearch). Halving the float width before indexing is often the cheapest memory win available.

For a catalog that changes daily rather than every second, GPU builds are rarely the bottleneck; the bigger ingestion cost is usually generating the embeddings themselves. Batch the embedding calls, skip documents whose embedded text has not changed, and re-embed everything only when you switch models. A GPU index build becomes interesting for full rebuilds of tens of millions of vectors, where it shortens the window in which a new index is being prepared.

The license story and what it means for you

In April 2021 AWS announced OpenSearch as a community fork of Elasticsearch and Kibana 7.10.2 under the Apache License 2.0 (AWS), after Elastic had moved to SSPL and its own Elastic License. On 29 August 2024 Elastic added the AGPL v3 as a third option next to ELv2 and SSPL (Elastic). OpenSearch remains plain Apache-2.0.

The practical consequence is in the feature list, not just the license text. Several of the Elasticsearch vector features covered above, DiskBBQ and GPU indexing among them, require an Enterprise subscription, while the matching OpenSearch features ship in the Apache-2.0 distribution. Managed offerings from cloud providers add their own terms on top, so compare the feature you actually need in the edition you would actually run.

When Elasticsearch or OpenSearch is the wrong vector store

A dedicated vector store wins when vectors are the main workload rather than an add-on to text search, when the index is too large to keep in page cache at a price you accept, or when you need per-tenant namespaces, very high write rates or vector-native features that the search engines do not offer. It also wins when nobody on the team wants to operate a JVM search cluster in the first place. In those cases, start from the dedicated options in pgvector or the comparison posts above.

Moving later is not expensive if you plan for it. Keep the embedding model, its version and the text that was embedded next to each vector, and write documents through one indexing service rather than from several places. Then switching the vector store, or adding a dedicated one for a single workload, means re-running ingestion against a new target instead of rebuilding the whole pipeline.

A practical checklist

1. Estimate vector memory at your dimension count and quantization level, and compare it with node page cache.
2. Put filters inside the kNN clause, not next to it.
3. Start with the version’s default quantization and keep rescoring on.
4. Fuse BM25 and vectors with RRF first, then test score normalization.
5. Measure recall at k with exact kNN as ground truth on a sample.
6. Check which features need a paid subscription before designing around them.
7. Re-test after every minor upgrade, because defaults change between versions.

If you want vector search added to a search cluster you already run, with recall measured rather than assumed, see my AI engineering work.

Sources

1. Elastic docs: kNN search
2. Elastic docs: filtered kNN search
3. Elastic docs: dense_vector field type
4. Elastic docs: BBQ
5. Elastic docs: reciprocal rank fusion
6. Elastic docs: GPU vector indexing
7. Elastic blog: Elasticsearch is open source again (Aug 2024)
8. OpenSearch docs: methods and engines
9. OpenSearch docs: memory-optimized vectors
10. OpenSearch docs: reciprocal rank fusion
11. OpenSearch docs: version history
12. AWS Open Source blog: introducing OpenSearch (Apr 2021)