[{"data":1,"prerenderedAt":737},["ShallowReactive",2],{"blog-lancedb-en":3},{"slug":4,"published":5,"minutes":6,"category":7,"tags":8,"keywords":13,"about":21,"sources":28,"cover":53,"og":54,"expertise":55,"locales":56,"lang":57,"title":60,"description":61,"coverAlt":62,"metaTitle":63,"takeaways":64,"faq":70,"toc":83,"blocks":108,"others":484},"lancedb","2026-09-21",9,"rag",[9,10,11,12],"Vector search","Hybrid search","Embedded database","RAG",[4,14,15,16,17,18,19,20],"lancedb review","lance vector database","embedded vector database","lancedb vs qdrant","hybrid search rrf","lancedb indexing","lance data format",[22,25],{"name":23,"url":24},"Vector database","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FVector_database",{"name":26,"url":27},"Apache Arrow","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FApache_Arrow",[29,32,35,38,41,44,47,50],{"title":30,"url":31},"LanceDB quickstart","https:\u002F\u002Fdocs.lancedb.com\u002Fquickstart",{"title":33,"url":34},"LanceDB vector indexes","https:\u002F\u002Fdocs.lancedb.com\u002Findexing\u002Fvector-index",{"title":36,"url":37},"LanceDB indexing guide","https:\u002F\u002Fdocs.lancedb.com\u002Findexing\u002Findex",{"title":39,"url":40},"LanceDB hybrid search","https:\u002F\u002Fdocs.lancedb.com\u002Fsearch\u002Fhybrid-search",{"title":42,"url":43},"LanceDB Enterprise","https:\u002F\u002Fdocs.lancedb.com\u002Fenterprise",{"title":45,"url":46},"LanceDB frequently asked questions","https:\u002F\u002Fdocs.lancedb.com\u002Ffaq\u002Ffaq-oss",{"title":48,"url":49},"LanceDB pricing","https:\u002F\u002Flancedb.com\u002Fpricing",{"title":51,"url":52},"LanceDB on PyPI","https:\u002F\u002Fpypi.org\u002Fproject\u002Flancedb\u002F","\u002Fimages\u002Fblog\u002Flancedb\u002Fcover.webp","\u002Fimages\u002Fblog\u002Flancedb\u002Fog.jpg","ai-engineer",[57,58,59],"en","de","hu","LanceDB: vector search that starts as a library","A review of LanceDB: an Apache-2.0 embedded vector library, its IVF and HNSW index choices, hybrid search with rank fusion, and what the Enterprise tier adds.","A nested box holding three sheets, two arrows merging from a square and a circle tray.","LanceDB review: embedded vector database · Balázs Csorba",[65,66,67,68,69],"LanceDB 0.40.0, published 7 October 2026, is an Apache-2.0 Rust library embedded in the application process, with Python, JavaScript and Rust clients and no server to run.","Below roughly a million rows a vector index is optional: the vendor's FAQ measures 100,000 pairs of 1,000-dimensional vectors at under 20 ms and recommends skipping the index for small tables.","HNSW is not a top-level index here — it exists only inside IVF partitions — and the docs warn that HNSW-backed indexes show higher latency variance under metadata filters.","Hybrid search is the strongest feature: BM25 full-text search and vector search merged by a reciprocal rank fusion reranker, with prefiltering on by default.","The vendor's own comparison puts the open-source build at 10 to 50 queries per second and 500 to 1,000 ms from object storage, against up to 10,000 queries and 50 to 200 ms on Enterprise.",[71,74,77,80],{"q":72,"a":73},"Do I need a vector index in LanceDB?","Not at first. The FAQ puts brute-force search at under 20 ms for 100,000 pairs of 1,000-dimensional vectors and says a vector index becomes worthwhile beyond roughly one million rows or higher dimensions; below that, scanning is usually fast enough.",{"q":75,"a":76},"Why is there no plain HNSW index?","In LanceDB, HNSW is a substructure inside IVF partitions rather than a top-level index, which combines IVF scalability with HNSW recall. The available types are IVF_HNSW_FLAT, IVF_HNSW_PQ and IVF_HNSW_SQ, alongside the unquantised and quantised IVF variants.",{"q":78,"a":79},"How do you keep an index healthy in the open-source build?","Index builds run asynchronously and appended rows stay outside the index until they are folded in with optimize(). create_index returns immediately, wait_for_index waits for the coverage to be complete, and fast_search() skips the slower fallback path over rows that are not indexed yet.",{"q":81,"a":82},"What does LanceDB Enterprise cost?","There is no public rate card: the pricing page is a contact form, and the Enterprise tier is sold as a managed deployment or bring-your-own-cloud with SOC 2 Type II and HIPAA coverage. The vendor's published benchmark works out at roughly $779 a month for 100 million vectors.",[84,87,90,93,96,99,102,105],{"id":85,"title":86},"what-it-is","What LanceDB is",{"id":88,"title":89},"how-it-works","How a query runs",{"id":91,"title":92},"indexes","Which index to build",{"id":94,"title":95},"getting-started","Getting started",{"id":97,"title":98},"where-it-shingles","Where it shingles",{"id":100,"title":101},"pricing","Pricing",{"id":103,"title":104},"verdict","Verdict",{"id":106,"title":107},"sources","Sources",[109,113,116,119,138,192,193,200,206,225,228,229,232,276,279,312,313,316,318,332,346,347,350,397,400,414,415,422,430,431,434,446,456,457],{"type":110,"content":111},"paragraph",[112],"LanceDB is an embedded vector database: a Rust library that links into the application process and stores vectors, metadata and source text in the Lance columnar format. There is no server to deploy, no cluster to size and no connection pool to tune — the strongest argument for it, and the reason it should be compared as much with SQLite as with Qdrant.",{"type":110,"content":114},[115],"It competes with the embedded options — SQLite with a vector extension, Chroma, an in-process index — and, once pointed at object storage, with the hosted services. It replaces the pattern of running a dedicated search cluster for a corpus that fits on one machine.",{"type":117,"level":118,"id":85,"text":86},"heading",2,{"type":110,"content":120},[121,122,126,127,126,130,133,134,137],"The current release is 0.40.0, published on 7 October 2026 and requiring Python 3.10 or newer, with JavaScript and Rust clients against the same Rust core. Data lives wherever the connection URI points: a local directory, ",{"tag":123,"children":124},"code",[125],"s3:\u002F\u002F",", ",{"tag":123,"children":128},[129],"gs:\u002F\u002F",{"tag":123,"children":131},[132],"az:\u002F\u002F",", or ",{"tag":123,"children":135},[136],"db:\u002F\u002F"," for the Enterprise cluster, and the same Lance files are readable by both editions.",{"type":139,"ordered":140,"items":141},"list",false,[142,148,153,158,177,182,187],[143,147],{"tag":144,"children":145},"strong",[146],"Licence:"," Apache-2.0 for the library; Enterprise is a commercial product sold as managed or bring-your-own-cloud.",[149,152],{"tag":144,"children":150},[151],"Shape:"," embedded in your process — no daemon, no sharding, no separate query language.",[154,157],{"tag":144,"children":155},[156],"Storage:"," the Lance format holds vectors, metadata and raw data in one table, with versioning and zero-copy reads through Apache Arrow.",[159,162,163,126,166,126,169,172,173,176],{"tag":144,"children":160},[161],"Indexes:"," ",{"tag":123,"children":164},[165],"IVF_RQ",{"tag":123,"children":167},[168],"IVF_PQ",{"tag":123,"children":170},[171],"IVF_HNSW_SQ"," and ",{"tag":123,"children":174},[175],"IVF_HNSW_FLAT"," for vectors, BM25 for full text, plus scalar indexes.",[178,181],{"tag":144,"children":179},[180],"Search:"," vector, full-text and hybrid queries with reranking, prefiltering by default, distance bounds and an exact-scan escape hatch.",[183,186],{"tag":144,"children":184},[185],"Scale guidance:"," comfortable on a single node; the FAQ targets roughly 10 to 50 billion rows and 10 to 30 TB before Enterprise is the answer.",[188,191],{"tag":144,"children":189},[190],"Clients:"," Python, JavaScript and Rust, installed with pip, npm or cargo.",{"type":117,"level":118,"id":88,"text":89},{"type":110,"content":194},[195,196,199],"A search without an index is a scan: every vector is compared against the query and the closest k are returned, which is exact and fast enough while the table is small. An IVF index spends training time clustering vectors into partitions, so a query compares against a few centroids first and then brute-forces inside those partitions; ",{"tag":123,"children":197},[198],"nprobes",", which defaults to 20, is how many partitions are opened. HNSW then sits inside each partition as a second-level graph, which is why the documentation can say that HNSW is not a top-level index in LanceDB.",{"type":201,"src":202,"alt":203,"wide":204,"caption":205},"image","\u002Fimages\u002Fblog\u002Flancedb\u002Fdiagram-7.webp","How one hybrid query runs A left to right flow: rows are ingested into a Lance table stored on local disk or object storage. The table feeds two indexes in parallel, a vector index over the embedding column and a BM25 full-text index over the text column, and a reciprocal rank fusion step merges both result lists into one ranked top-k.",true,"A hybrid search: one Lance table feeds a vector index and a BM25 index, and rank fusion merges both into one ranked list.",{"type":110,"content":207},[208,209,212,213,216,217,220,221,224],"Full-text search is a separate BM25 index built with ",{"tag":123,"children":210},[211],"create_fts_index",", and hybrid search runs both halves and merges them: by default with an RRF reranker, which turns each list into ranks and adds them, so a result that ranks well in either half wins without either score scale dominating. Filters passed to ",{"tag":123,"children":214},[215],"where"," are prefilters by default, applied before scoring; ",{"tag":123,"children":218},[219],"prefilter=False"," moves the filter after the sub-queries, which can return fewer than ",{"tag":123,"children":222},[223],"limit"," rows.",{"type":110,"content":226},[227],"The storage layer decides the latency profile. On local disk reads are memory-mapped and quick; pointed at S3, GCS or Azure Blob, every cold read is a network round trip, and the vendor's own comparison puts that at 500 to 1,000 ms for the open-source build against 50 to 200 ms on Enterprise, where an NVMe cache absorbs the repeat reads.",{"type":117,"level":118,"id":91,"text":92},{"type":110,"content":230},[231],"The index choice is a compression decision first, and the documentation is direct about the trade:",{"type":233,"head":234,"rows":243},"table",[235,237,239,241],[236],"Priority",[238],"Index",[240],"Compression",[242],"Note",[244,252,260,268],[245,247,248,250],[246],"Maximum compression",[165],[249],"About 1\u002F32 of raw size",[251],"RaBitQ quantisation over IVF",[253,255,256,258],[254],"Accuracy at 256 dimensions or fewer",[168],[257],"1\u002F64 to 1\u002F16 of raw size",[259],"Product quantisation, recall tuned with refine_factor",[261,263,264,266],[262],"Best recall-to-latency trade-off",[171],[265],"A little over 1\u002F4 of raw size",[267],"IVF partitions with HNSW inside, scalar quantisation",[269,271,272,274],[270],"Highest recall, no quantisation",[175],[273],"Raw size plus graph overhead",[275],"The expensive, faithful option",{"type":110,"content":277},[278],"Two rules from the docs matter more than the table. HNSW never appears alone: it is only ever a substructure inside IVF partitions, so there is no plain HNSW index to create. And if the workload carries metadata filters, the docs tell you to prefer IVF_RQ or IVF_PQ, because the HNSW-backed variants show higher latency variance in filtered searches.",{"type":280,"variant":281,"body":282,"title":311},"callout","warn",[283],[284,285,172,287,289,290,292,293,295,296,299,300,303,304,307,308,310],"For ",{"tag":123,"children":286},[165],{"tag":123,"children":288},[168]," the guidance is to keep ",{"tag":123,"children":291},[198]," at the default and raise it only when recall falls short; for the IVF_HNSW variants to keep ",{"tag":123,"children":294},[198]," and tune ",{"tag":123,"children":297},[298],"ef"," first, starting around 1.5 times k and going up to 10 times k. ",{"tag":123,"children":301},[302],"refine_factor"," should sit between 5 and 50, and a filter that matches few rows may need ",{"tag":123,"children":305},[306],"maximum_nprobes"," raised to return ",{"tag":123,"children":309},[223]," results at all.","Tune nprobes before ef",{"type":117,"level":118,"id":94,"text":95},{"type":110,"content":314},[315],"Install the package, point at a directory and the table is on disk. The snippet below builds both indexes and runs the hybrid query the documentation recommends, with the metadata filter applied before scoring.",{"type":123,"code":317},"import lancedb\nfrom lancedb.rerankers import RRFReranker\n\ndb = lancedb.connect(\".\u002Fdata\")            # a local directory, no server\ntable = db.open_table(\"documents\")\ntable.create_fts_index(\"text\")            # BM25, built in the background\n\nresults = (\n    table.search(query_type=\"hybrid\")\n    .vector(embed(query))                 # your embedding model\n    .text(\"refunds within 14 days\")\n    .where(\"lang = 'en'\", prefilter=True)  # default: filter before scoring\n    .rerank(RRFReranker())                # the default hybrid reranker\n    .limit(5)\n    .to_list()\n)\nfor row in results:\n    print(round(row[\"_relevance_score\"], 3), row[\"text\"][:80])\n",{"type":110,"content":319},[320,321,323,324,327,328,331],"Nothing in that snippet needs a server, a container or an API key, and the same code runs against ",{"tag":123,"children":322},[125]," by changing the connect URI. Full-text and vector index builds return immediately and finish in the background; ",{"tag":123,"children":325},[326],"wait_for_index"," together with ",{"tag":123,"children":329},[330],"index_stats"," is how a job checks that nothing is left unindexed.",{"type":280,"variant":333,"body":334,"title":345},"note",[335],[336,337,340,341,344],"New rows stay outside an existing index until they are folded in with ",{"tag":123,"children":338},[339],"optimize()",", and until then a normal search pays for a slower fallback scan while ",{"tag":123,"children":342},[343],"fast_search()"," skips it. In the open-source build that schedule is yours: compaction and reindexing are calls someone has to book somewhere.","Rows appended after the build",{"type":117,"level":118,"id":97,"text":98},{"type":110,"content":348},[349],"The weaknesses follow from the architecture. One process means one host: the vendor's own comparison caps the open-source build at 10 to 50 queries per second with no cache, and every maintenance task — compaction, reindexing, index fragmentation after deletes — is a job someone has to schedule. Concurrent writes are bounded by how many times a writer will retry a commit, and Python users are told not to fork.",{"type":233,"head":351,"rows":360},[352,354,356,358],[353],"Engine",[355],"Licence",[357],"Index options",[359],"Operational shape",[361,370,379,388],[362,364,366,368],[363],"LanceDB",[365],"Apache-2.0, embedded",[367],"IVF and IVF-HNSW, BM25, scalar",[369],"A library inside your process",[371,373,375,377],[372],"Qdrant",[374],"Apache-2.0, server",[376],"Filterable HNSW, scalar and product quantisation",[378],"A container or a managed cloud",[380,382,384,386],[381],"pgvector",[383],"PostgreSQL licence",[385],"HNSW and IVFFlat inside Postgres",[387],"An extension in a database you already run",[389,391,393,395],[390],"Weaviate",[392],"BSD-3-Clause",[394],"HNSW, flat and dynamic",[396],"Single binary, optional cluster",{"type":110,"content":398},[399],"The honest summary: LanceDB is the cheapest thing to run and the most maintenance to own. The vendor's table puts single-process throughput at 10 to 50 queries per second and object-storage latency at 500 to 1,000 ms, with distributed search and platform-managed compaction still marked as coming soon on the Enterprise side — worth knowing before a purchase decision rests on the roadmap.",{"type":110,"content":401},[402,403,406,407,172,410,413],"API asymmetry is the other papercut: on an Enterprise ",{"tag":123,"children":404},[405],"RemoteTable"," the table-level ",{"tag":123,"children":408},[409],"to_arrow",{"tag":123,"children":411},[412],"to_pandas"," calls are refused, so materialisation has to go through the query builder, and an operation can fail on a service-level policy instead of on the data. Code that moves from OSS to Enterprise is close to the same, but not the same.",{"type":117,"level":118,"id":100,"text":101},{"type":110,"content":416},[417,418,421],"The library is Apache-2.0 and free in every sense that matters: ",{"tag":123,"children":419},[420],"pip install",", no account, no metering. The pricing page does not list a rate card — it is a contact form — and the Enterprise tier is sold as managed or bring-your-own-cloud with SOC 2 Type II, HIPAA coverage and OpenTelemetry metrics and traces. The one concrete number the vendor publishes is a benchmark of roughly $779 a month for 100 million vectors.",{"type":139,"ordered":140,"items":423},[424,426,428],[425],"Open source: the library, the indexes, hybrid search and the CLI maintenance calls, under Apache-2.0, with community support.",[427],"Enterprise: managed or BYOC, distributed query nodes, an NVMe cache, platform-run indexing and compaction, and compliance under SOC 2 Type II and HIPAA.",[429],"Coming soon, in the vendor's own table: distributed search, distributed indexing and compaction.",{"type":117,"level":118,"id":103,"text":104},{"type":110,"content":432},[433],"LanceDB is the right answer when the corpus and the traffic fit one machine, and the wrong answer when they do not — the vendor says as much in its own comparison table. Its strength is that retrieval becomes a library call instead of an infrastructure project, and its cost is that every operational duty of a database lands on the application team. The opinionated reading: for a RAG pipeline under a million chunks, running a separate vector database is an unnecessary service to operate, and LanceDB is what that pipeline should reach for first.",{"type":139,"ordered":204,"items":435},[436,438,440,442,444],[437],"Use it for a prototype, a single-node RAG pipeline or an edge deployment, where a database server would be the heaviest component in the stack.",[439],"Use it when the data is already in object storage and the Lance format's versioning and zero-copy reads remove a second copy of the corpus.",[441],"Think twice when queries must stay under 100 ms from S3 with real concurrency — the vendor's own figure for OSS is 500 to 1,000 ms and 10 to 50 queries per second.",[443],"Budget for maintenance: optimize, compaction and reindexing are unscheduled work in the open-source build, and index fragmentation grows with every delete.",[445],"Choose the Enterprise tier only for the distributed parts; the API differences, from RemoteTable materialisation limits to cluster-side guardrails, are what lock the application in.",{"type":280,"variant":447,"body":448,"title":455},"tip",[449],[450,451,454],"Build the index on real data and measure recall against ",{"tag":123,"children":452},[453],"bypass_vector_index()",", which runs an exact scan and gives the ground truth. The gap between the two answers is the actual cost of the approximation, and it is the number a latency budget should be written from.","One measurement before committing",{"type":117,"level":118,"id":106,"text":107},{"type":139,"ordered":204,"items":458},[459,463,466,469,472,475,478,481],[460],{"tag":461,"href":31,"children":462},"a",[30],[464],{"tag":461,"href":34,"children":465},[33],[467],{"tag":461,"href":37,"children":468},[36],[470],{"tag":461,"href":40,"children":471},[39],[473],{"tag":461,"href":43,"children":474},[42],[476],{"tag":461,"href":46,"children":477},[45],[479],{"tag":461,"href":49,"children":480},[48],[482],{"tag":461,"href":52,"children":483},[51],[485,558,618,676],{"slug":486,"published":487,"minutes":6,"category":7,"tags":488,"keywords":492,"about":505,"sources":515,"cover":552,"og":553,"expertise":55,"locales":554,"lang":57,"title":555,"description":556,"coverAlt":557},"embedding-model-selection-2026","2026-10-17",[489,12,490,9,491],"Embeddings","MTEB","Model selection",[493,494,495,496,497,498,499,500,501,502,503,504],"best embedding model 2026","how to choose an embedding model","embedding model comparison","MTEB leaderboard explained","RTEB benchmark","open source embedding model","multilingual embedding model","voyage-4 vs openai embeddings","Qwen3-Embedding","EmbeddingGemma","embedding model for RAG","switching embedding models re-embed",[506,509,512],{"name":507,"url":508},"Word embedding","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FWord_embedding",{"name":510,"url":511},"Information retrieval","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FInformation_retrieval",{"name":513,"url":514},"Discounted cumulative gain","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FDiscounted_cumulative_gain",[516,519,522,525,528,531,534,537,540,543,546,549],{"title":517,"url":518},"Enevoldsen et al.: MMTEB, Massive Multilingual Text Embedding Benchmark (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.13595",{"title":520,"url":521},"Hugging Face: Introducing RTEB, a new standard for retrieval evaluation (Oct 2025)","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Frteb",{"title":523,"url":524},"OpenAI: Vector embeddings guide","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fembeddings",{"title":526,"url":527},"OpenAI: API pricing","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fpricing",{"title":529,"url":530},"Voyage AI: The Voyage 4 model family, shared embedding space with MoE architecture (Jan 2026)","https:\u002F\u002Fblog.voyageai.com\u002F2026\u002F01\u002F15\u002Fvoyage-4\u002F",{"title":532,"url":533},"Voyage AI: Text embeddings documentation","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fembeddings",{"title":535,"url":536},"Voyage AI: Pricing","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fpricing",{"title":538,"url":539},"Google: Gemini API embeddings documentation","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fembeddings",{"title":541,"url":542},"Google: Gemini API pricing","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fpricing",{"title":544,"url":545},"Cohere: Embed models documentation","https:\u002F\u002Fdocs.cohere.com\u002Fdocs\u002Fcohere-embed",{"title":547,"url":548},"Qwen: Qwen3-Embedding-8B model card","https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-Embedding-8B",{"title":550,"url":551},"Google Developers Blog: Introducing EmbeddingGemma (Sep 2025)","https:\u002F\u002Fdevelopers.googleblog.com\u002Fen\u002Fintroducing-embeddinggemma\u002F","\u002Fimages\u002Fblog\u002Fembedding-model-selection-2026\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fembedding-model-selection-2026\u002Fog.jpg",[57,58,59],"Choosing an embedding model in 2026: leaderboards, costs and a bake-off on your own data","How to pick an embedding model for RAG in 2026: what MTEB and RTEB tell you, hosted vs open weights, prices, dimensions, and a one-week bake-off.","Three columns of stacked blocks, the middle tallest, each measured by a short bar and a wide bracket.",{"slug":559,"published":487,"minutes":560,"category":7,"tags":561,"keywords":565,"about":577,"sources":587,"cover":612,"og":613,"expertise":55,"locales":614,"lang":57,"title":615,"description":616,"coverAlt":617},"contextual-retrieval-late-chunking",10,[12,562,489,563,564],"Chunking","Contextual retrieval","Late chunking",[566,567,568,569,570,571,572,573,574,575,576],"contextual retrieval","anthropic contextual retrieval","late chunking","contextual chunk embeddings","contextual embeddings rag","contextual bm25","chunk context rag","voyage-context-3","voyage-context-4","late chunking vs contextual retrieval","rag chunking context loss",[578,581,584],{"name":579,"url":580},"Retrieval-augmented generation","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FRetrieval-augmented_generation",{"name":582,"url":583},"Okapi BM25","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FOkapi_BM25",{"name":585,"url":586},"Sentence embedding","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FSentence_embedding",[588,591,594,597,600,603,606,609],{"title":589,"url":590},"Anthropic: Introducing Contextual Retrieval (Sep 2024)","https:\u002F\u002Fwww.anthropic.com\u002Fnews\u002Fcontextual-retrieval",{"title":592,"url":593},"Günther et al.: Late Chunking, Contextual Chunk Embeddings Using Long-Context Embedding Models (2024, rev. 2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2409.04701",{"title":595,"url":596},"Jina AI: Late Chunking in Long-Context Embedding Models (Aug 2024)","https:\u002F\u002Fjina.ai\u002Fnews\u002Flate-chunking-in-long-context-embedding-models\u002F",{"title":598,"url":599},"Voyage AI: Introducing voyage-context-3 (Jul 2025)","https:\u002F\u002Fblog.voyageai.com\u002F2025\u002F07\u002F23\u002Fvoyage-context-3\u002F",{"title":601,"url":602},"Voyage AI: voyage-context-4 (Jun 2026)","https:\u002F\u002Fblog.voyageai.com\u002F2026\u002F06\u002F29\u002Fvoyage-context-4\u002F",{"title":604,"url":605},"Voyage AI: Contextualized chunk embeddings documentation","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fcontextualized-chunk-embeddings",{"title":607,"url":608},"Anthropic: Claude API pricing","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fabout-claude\u002Fpricing",{"title":610,"url":611},"Qu, Tu and Bao: Is Semantic Chunking Worth the Computational Cost? (2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.13070","\u002Fimages\u002Fblog\u002Fcontextual-retrieval-late-chunking\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fcontextual-retrieval-late-chunking\u002Fog.jpg",[57,58,59],"Contextual retrieval and late chunking: giving RAG chunks back their context","Chunks lose their document. How contextual retrieval, late chunking and contextualized embedding models fix that, what each costs, and when to skip it.","Four loose square pieces gathered by a wide bracket below, each joined upward to a large sheet.",{"slug":619,"published":487,"minutes":6,"category":7,"tags":620,"keywords":624,"about":637,"sources":645,"cover":670,"og":671,"expertise":55,"locales":672,"lang":57,"title":673,"description":674,"coverAlt":675},"rag-access-control-permissions",[12,621,622,623,9],"Access control","Security","Multi-tenancy",[625,626,627,628,629,630,631,632,633,634,635,636],"rag access control","permission aware rag","document level security rag","rag multi tenancy","vector database access control","pgvector row level security","security trimming azure ai search","openfga rag authorization","qdrant multitenancy","pinecone namespaces multitenancy","owasp llm08 vector and embedding weaknesses","rag data leakage",[638,641,644],{"name":639,"url":640},"Access-control list","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FAccess-control_list",{"name":642,"url":643},"Multitenancy","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FMultitenancy",{"name":579,"url":580},[646,649,652,655,658,661,664,667],{"title":647,"url":648},"OWASP GenAI Security Project: LLM08:2025 Vector and Embedding Weaknesses","https:\u002F\u002Fgenai.owasp.org\u002Fllmrisk\u002Fllm082025-vector-and-embedding-weaknesses\u002F",{"title":650,"url":651},"PostgreSQL documentation: Row Security Policies","https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002Fcurrent\u002Fddl-rowsecurity.html",{"title":653,"url":654},"pgvector README: filtering and iterative index scans","https:\u002F\u002Fgithub.com\u002Fpgvector\u002Fpgvector",{"title":656,"url":657},"Qdrant: Multitenancy and partitions guide","https:\u002F\u002Fqdrant.tech\u002Fdocumentation\u002Fguides\u002Fmultiple-partitions\u002F",{"title":659,"url":660},"Pinecone: Implement multitenancy using namespaces","https:\u002F\u002Fdocs.pinecone.io\u002Fguides\u002Findex-data\u002Fimplement-multitenancy",{"title":662,"url":663},"Microsoft Learn: Document-level access control in Azure AI Search","https:\u002F\u002Flearn.microsoft.com\u002Fen-us\u002Fazure\u002Fsearch\u002Fsearch-document-level-access-overview",{"title":665,"url":666},"OpenFGA: RAG authorization use case","https:\u002F\u002Fopenfga.dev\u002Fdocs\u002Fuse-cases\u002Frag-authorization",{"title":668,"url":669},"Morris et al.: Text Embeddings Reveal (Almost) As Much As Text (EMNLP 2023)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2310.06816","\u002Fimages\u002Fblog\u002Frag-access-control-permissions\u002Fcover.webp","\u002Fimages\u002Fblog\u002Frag-access-control-permissions\u002Fog.jpg",[57,58,59],"Access control for RAG: permission-aware retrieval without leaking documents","How RAG assistants leak documents and how to stop it: ACLs on every chunk, pre-filtering, row-level security in pgvector, permission sync and leak tests.","Seven cards in a row, six with a lock badge, one bare card sending an arrow outward.",{"slug":677,"published":487,"minutes":560,"category":7,"tags":678,"keywords":683,"about":695,"sources":703,"cover":730,"og":731,"expertise":732,"locales":733,"lang":57,"title":734,"description":735,"coverAlt":736},"rag-product-catalog-pim-data",[12,679,680,681,682],"PIM","Product data","B2B e-commerce","Metadata filtering",[684,685,686,687,688,689,690,691,692,693,694],"rag product catalog","rag pim data","llm product data","product catalog chatbot","akeneo rag","pimcore ai search","rag structured data","metadata filtering rag","product attributes embeddings","b2b product assistant","etim classification ai",[696,699,700],{"name":697,"url":698},"Product information management","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FProduct_information_management",{"name":579,"url":580},{"name":701,"url":702},"ETIM (standard)","https:\u002F\u002Fwww.etim-international.com\u002Fclassification\u002F",[704,707,710,713,715,718,721,724,727],{"title":705,"url":706},"Akeneo API: Products, product values, metric values and product models","https:\u002F\u002Fapi.akeneo.com\u002Fconcepts\u002Fproducts.html",{"title":708,"url":709},"Akeneo Help Center: What about products with variants?","https:\u002F\u002Fhelp.akeneo.com\u002Fserenity-take-the-power-over-your-products\u002Fserenity-what-about-products-with-variants",{"title":711,"url":712},"Pimcore documentation: Classification Store","https:\u002F\u002Fdocs.pimcore.com\u002Fplatform\u002FPimcore\u002FObjects\u002FObject_Classes\u002FData_Types\u002FClassification_Store\u002F",{"title":714,"url":702},"ETIM International: Classification",{"title":716,"url":717},"ETIM International: ETIM releases and the IXF format","https:\u002F\u002Fetim-international.com\u002F?p=754",{"title":719,"url":720},"Reddy et al.: Shopping Queries Dataset, a large-scale ESCI benchmark for improving product search (2022)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2206.06588",{"title":722,"url":723},"Amazon Science: esci-data repository","https:\u002F\u002Fgithub.com\u002Famazon-science\u002Fesci-data",{"title":725,"url":726},"Qdrant documentation: Filtering","https:\u002F\u002Fqdrant.tech\u002Fdocumentation\u002Fconcepts\u002Ffiltering\u002F",{"title":728,"url":729},"Sui et al.: Table Meets LLM, can large language models understand structured table data? (WSDM 2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.13062","\u002Fimages\u002Fblog\u002Frag-product-catalog-pim-data\u002Fcover.webp","\u002Fimages\u002Fblog\u002Frag-product-catalog-pim-data\u002Fog.jpg","b2b-ecommerce-developer",[57,58,59],"RAG over product catalogues: turning PIM data into answers that hold","Product data is structured, so chunking it like PDFs fails. How to serialize PIM attributes, route spec questions to filters and keep answers exact.","A frame of nine squares with tiny symbols, funnelling down to one card.",1792261155819]