[{"data":1,"prerenderedAt":697},["ShallowReactive",2],{"blog-contextual-retrieval-late-chunking-en":3},{"slug":4,"published":5,"minutes":6,"category":7,"tags":8,"keywords":14,"about":26,"sources":36,"cover":61,"og":62,"expertise":63,"locales":64,"lang":65,"title":68,"description":69,"coverAlt":70,"metaTitle":71,"takeaways":72,"faq":78,"toc":94,"blocks":125,"others":458},"contextual-retrieval-late-chunking","2026-10-17",10,"rag",[9,10,11,12,13],"RAG","Chunking","Embeddings","Contextual retrieval","Late chunking",[15,16,17,18,19,20,21,22,23,24,25],"contextual retrieval","anthropic contextual retrieval","late chunking","contextual chunk embeddings","contextual embeddings rag","contextual bm25","chunk context rag","voyage-context-3","voyage-context-4","late chunking vs contextual retrieval","rag chunking context loss",[27,30,33],{"name":28,"url":29},"Retrieval-augmented generation","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FRetrieval-augmented_generation",{"name":31,"url":32},"Okapi BM25","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FOkapi_BM25",{"name":34,"url":35},"Sentence embedding","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FSentence_embedding",[37,40,43,46,49,52,55,58],{"title":38,"url":39},"Anthropic: Introducing Contextual Retrieval (Sep 2024)","https:\u002F\u002Fwww.anthropic.com\u002Fnews\u002Fcontextual-retrieval",{"title":41,"url":42},"Günther et al.: Late Chunking, Contextual Chunk Embeddings Using Long-Context Embedding Models (2024, rev. 2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2409.04701",{"title":44,"url":45},"Jina AI: Late Chunking in Long-Context Embedding Models (Aug 2024)","https:\u002F\u002Fjina.ai\u002Fnews\u002Flate-chunking-in-long-context-embedding-models\u002F",{"title":47,"url":48},"Voyage AI: Introducing voyage-context-3 (Jul 2025)","https:\u002F\u002Fblog.voyageai.com\u002F2025\u002F07\u002F23\u002Fvoyage-context-3\u002F",{"title":50,"url":51},"Voyage AI: voyage-context-4 (Jun 2026)","https:\u002F\u002Fblog.voyageai.com\u002F2026\u002F06\u002F29\u002Fvoyage-context-4\u002F",{"title":53,"url":54},"Voyage AI: Contextualized chunk embeddings documentation","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fcontextualized-chunk-embeddings",{"title":56,"url":57},"Anthropic: Claude API pricing","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fabout-claude\u002Fpricing",{"title":59,"url":60},"Qu, Tu and Bao: Is Semantic Chunking Worth the Computational Cost? (2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.13070","\u002Fimages\u002Fblog\u002Fcontextual-retrieval-late-chunking\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fcontextual-retrieval-late-chunking\u002Fog.jpg","ai-engineer",[65,66,67],"en","de","hu","Contextual retrieval and late chunking: giving RAG chunks back their context","Chunks lose their document. How contextual retrieval, late chunking and contextualized embedding models fix that, what each costs, and when to skip it.","Four loose square pieces gathered by a wide bracket below, each joined upward to a large sheet.","Contextual retrieval and late chunking · Balázs Csorba",[73,74,75,76,77],"Anthropic's contextual retrieval prepends 50 to 100 tokens of LLM-written context to each chunk and cut top-20 retrieval failures by 35%, by 49% with contextual BM25 and by 67% with a reranker.","With prompt caching the context generation is a one-off index cost: about $1.02 per million document tokens in 2024, roughly $0.38 to $3.84 at today's Haiku list prices by my arithmetic.","Late chunking embeds the whole document first and pools token vectors per chunk; it needs no LLM, but a long-context embedding model whose token outputs you can access.","In the late chunking paper the average gain over naive chunking was about 1.5 to 1.9 nDCG@10 points, largest with small chunks and absent when the surrounding text is unrelated.","Start with the cheap baseline, document title and section path in front of each chunk, and measure before paying for any of the three.",[79,82,85,88,91],{"q":80,"a":81},"What is contextual retrieval?","A preprocessing step published by Anthropic in September 2024. An LLM reads the whole document and each chunk, writes a short context that situates the chunk, and that context is prepended to the chunk before it is embedded and indexed for BM25.",{"q":83,"a":84},"What is late chunking?","A method from Jina AI researchers: run the embedding model over the whole document, then split the token embeddings into chunks and mean-pool each chunk. Every chunk vector is computed with the rest of the document in view, without any extra LLM call.",{"q":86,"a":87},"Is contextual retrieval expensive?","It costs one LLM call per chunk at indexing time, not per query. With prompt caching on the document, Anthropic estimated about $1.02 per million document tokens in 2024; recalculated with current Claude Haiku list prices it lands roughly between $0.38 and $3.84.",{"q":89,"a":90},"Does late chunking help keyword search?","No. Late chunking changes only the vectors. BM25 still sees the raw chunk text. Contextual retrieval changes the text itself, so it improves BM25 and vector search at the same time.",{"q":92,"a":93},"When should I skip all of this?","When chunks are already self-contained, such as product records or FAQ entries, or when the whole knowledge base fits in the prompt. Anthropic suggests that below about 200,000 tokens, putting everything in context can be simpler than RAG.",[95,98,101,104,107,110,113,116,119,122],{"id":96,"title":97},"context-loss","Why do chunks lose their context?",{"id":99,"title":100},"contextual-retrieval","How does contextual retrieval work?",{"id":102,"title":103},"cost","What does contextualization cost?",{"id":105,"title":106},"late-chunking","How does late chunking work?",{"id":108,"title":109},"contextual-models","What about contextualized embedding APIs?",{"id":111,"title":112},"comparison","How do the three approaches compare?",{"id":114,"title":115},"when-to-skip","When is added context not worth it?",{"id":117,"title":118},"implementation","What should the implementation get right?",{"id":120,"title":121},"which-to-pick","Which one should you pick?",{"id":123,"title":124},"sources","Sources",[126,130,139,142,145,148,149,152,161,164,202,205,208,209,212,274,277,278,281,287,290,293,295,296,299,301,304,305,356,368,369,397,398,426,427,430,431],{"type":127,"content":128},"paragraph",[129],"Chunking is a lossy step. A chunk that says \"the maximum load is 40 kg\" was perfectly clear inside its manual, where the heading two pages earlier named the product. Cut out and embedded alone, it no longer mentions the product, the variant or the document it came from, and a query that names the product may never find it.",{"type":127,"content":131},[132,133,138],"Three families of fixes appeared since 2024: contextual retrieval, which rewrites each chunk with an LLM; late chunking, which embeds the whole document before splitting; and contextualized embedding models sold as an API. This post compares what each changes, what the published numbers say, what they cost, and the cheap baseline I try first. It builds on the ",{"tag":134,"to":135,"children":136},"link","\u002Fblog\u002Frag-pipeline-chunking-hybrid-search-reranking",[137],"chunking and hybrid search pipeline"," described earlier.",{"type":140,"level":141,"id":96,"text":97},"heading",2,{"type":127,"content":143},[144],"Embedding models see only the text you pass them. When a document is split first and each piece is embedded separately, references that point outside the chunk, such as \"it\", \"this variant\", \"the above table\" or an implicit product name, carry no meaning. BM25 suffers the same way: the keyword that would match is in the heading, not in the chunk.",{"type":127,"content":146},[147],"Bigger chunks are the obvious workaround, but they dilute the vector and send more irrelevant text to the model. Smarter boundaries help less than one would hope: a 2024 study by Qu, Tu and Bao concluded that the extra cost of semantic chunking is not justified by consistent gains over fixed-size chunks. The fixes below leave boundaries alone and add context instead.",{"type":140,"level":141,"id":99,"text":100},{"type":127,"content":150},[151],"Anthropic published contextual retrieval on 19 September 2024. For every chunk, a small model receives the full document and the chunk and is asked for a short context that situates the chunk for search, answering with the context and nothing else. The result, usually 50 to 100 tokens, is prepended to the chunk. The combined text is then embedded and also indexed for BM25, which Anthropic calls contextual embeddings and contextual BM25.",{"type":153,"attrs":154,"inner":158,"caption":159},"diagram",{"viewBox":155,"role":156,"aria-labelledby":157},"0 0 720 170","img","crl1-t crl1-d","\u003Ctitle id=\"crl1-t\">Contextual retrieval at indexing time\u003C\u002Ftitle>\u003Cdesc id=\"crl1-d\">The full document is cached once. For each chunk a small LLM writes a short context, which is prepended to the chunk. The combined text goes to both the embedding index and the BM25 index; queries then use hybrid search and a reranker.\u003C\u002Fdesc>\u003Cdefs>\u003Cmarker id=\"crl1-ah\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\">\u003Cpath d=\"M0 0L10 5L0 10z\" class=\"d-head\" \u002F>\u003C\u002Fmarker>\u003C\u002Fdefs>\u003Crect x=\"20\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"79\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Document\u003C\u002Ftext>\u003Ctext x=\"79\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">cached prompt\u003C\u002Ftext>\u003Cpath d=\"M138 85H158\" class=\"d-line\" marker-end=\"url(#crl1-ah)\" \u002F>\u003Crect x=\"160\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"220\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Chunk\u003C\u002Ftext>\u003Ctext x=\"220\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">one per call\u003C\u002Ftext>\u003Cpath d=\"M279 85H299\" class=\"d-line\" marker-end=\"url(#crl1-ah)\" \u002F>\u003Crect x=\"301\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-accent\" \u002F>\u003Ctext x=\"360\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Small LLM\u003C\u002Ftext>\u003Ctext x=\"360\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">50–100 tokens\u003C\u002Ftext>\u003Cpath d=\"M419 85H439\" class=\"d-line\" marker-end=\"url(#crl1-ah)\" \u002F>\u003Crect x=\"441\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-gold\" \u002F>\u003Ctext x=\"500\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Context + chunk\u003C\u002Ftext>\u003Ctext x=\"500\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">one text\u003C\u002Ftext>\u003Cpath d=\"M560 85H580\" class=\"d-line\" marker-end=\"url(#crl1-ah)\" \u002F>\u003Crect x=\"582\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-mint\" \u002F>\u003Ctext x=\"641\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Two indexes\u003C\u002Ftext>\u003Ctext x=\"641\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">vectors + BM25\u003C\u002Ftext>",[160],"The LLM runs once per chunk at indexing time; nothing changes on the query path.",{"type":127,"content":162},[163],"The reported effect is large. Anthropic measured the top-20 retrieval failure rate, one minus recall@20, averaged over codebases, fiction, arXiv papers and science papers:",{"type":165,"head":166,"rows":173},"table",[167,169,171],[168],"Configuration",[170],"Failure rate",[172],"Reduction",[174,181,188,195],[175,177,179],[176],"Baseline embeddings",[178],"5.7%",[180],"–",[182,184,186],[183],"Contextual embeddings",[185],"3.7%",[187],"35%",[189,191,193],[190],"Contextual embeddings + contextual BM25",[192],"2.9%",[194],"49%",[196,198,200],[197],"Both, plus reranking",[199],"1.9%",[201],"67%",{"type":127,"content":203},[204],"Two details from the same post are easy to miss. Generic document summaries prepended to chunks gave very limited gains, so the context must be chunk-specific. And the reranked setup retrieved about 150 candidates, reranked them and kept the top 20, which performed better than passing 5 or 10 chunks.",{"type":206,"code":207},"code","import anthropic\n\nclient = anthropic.Anthropic()\n\ndef chunk_context(document: str, chunk: str) -> str:\n    \"\"\"One call per chunk; the document block is cached and reused for every chunk.\"\"\"\n    msg = client.messages.create(\n        model=\"claude-haiku-4-5\",\n        max_tokens=150,\n        messages=[{\"role\": \"user\", \"content\": [\n            {\"type\": \"text\", \"text\": f\"\u003Cdocument>\\n{document}\\n\u003C\u002Fdocument>\",\n             \"cache_control\": {\"type\": \"ephemeral\"}},\n            {\"type\": \"text\", \"text\": (\n                f\"Here is a chunk from the document:\\n\u003Cchunk>\\n{chunk}\\n\u003C\u002Fchunk>\\n\"\n                \"Write a short context that situates this chunk within the document \"\n                \"to improve search retrieval. Answer only with the context.\")},\n        ]}],\n    )\n    return msg.content[0].text.strip()\n\n# Index context and chunk together, but keep both fields for display and re-runs.\nrecord = {\"context\": ctx, \"chunk\": chunk, \"index_text\": f\"{ctx}\\n\\n{chunk}\"}",{"type":140,"level":141,"id":102,"text":103},{"type":127,"content":210},[211],"Anthropic estimated about $1.02 per million document tokens, one-off, assuming 8,000-token documents, 800-token chunks, 50 tokens of instructions and 100 tokens of context per chunk, with the document cached. That figure used Claude 3 Haiku prices from 2024. I recalculated it with the same assumptions and the current list prices from Anthropic's pricing page: per million document tokens that is 125 documents and 1,250 chunks, one cache write per document, nine cache reads, the chunk and instructions as normal input, and the context as output.",{"type":165,"head":213,"rows":222},[214,216,218,220],[215],"Item per 1M document tokens",[217],"Tokens",[219],"Haiku 4.5",[221],"Haiku 5.5 (prompts up to 100k)",[223,232,241,250,259],[224,226,228,230],[225],"Cache writes, documents",[227],"1.0M",[229],"$1.25",[231],"$0.13",[233,235,237,239],[234],"Cache reads, documents",[236],"9.0M",[238],"$0.90",[240],"$0.09",[242,244,246,248],[243],"Input, chunk and instructions",[245],"1.06M",[247],"$1.06",[249],"$0.11",[251,253,255,257],[252],"Output, context",[254],"0.125M",[256],"$0.63",[258],"$0.06",[260,265,266,270],[261],{"tag":262,"children":263},"strong",[264],"Total",[],[267],{"tag":262,"children":268},[269],"$3.84",[271],{"tag":262,"children":272},[273],"$0.38",{"type":127,"content":275},[276],"Either way it is an indexing cost, paid again only when a document changes, and small next to the engineering time. The catch is change: when one paragraph of a document changes, the context of every chunk in it may change too, so re-contextualize the whole document, not only the edited chunk.",{"type":140,"level":141,"id":105,"text":106},{"type":127,"content":279},[280],"Late chunking, proposed by Günther and colleagues at Jina AI in September 2024, moves the split to the end. The long-context embedding model reads the whole document, or as much as fits, and produces one vector per token. Each token vector already reflects the surrounding text through attention. Only then are the chunk boundaries applied: the token vectors inside each chunk are mean-pooled into one chunk embedding. The paper's phrase is chunking applied after the transformer and just before mean pooling.",{"type":153,"attrs":282,"inner":284,"caption":285},{"viewBox":155,"role":156,"aria-labelledby":283},"crl2-t crl2-d","\u003Ctitle id=\"crl2-t\">Naive chunking versus late chunking\u003C\u002Ftitle>\u003Cdesc id=\"crl2-d\">In late chunking the whole document passes through the transformer once. The token embeddings are then split at the chunk boundaries and mean-pooled, so each chunk vector carries context from the rest of the document.\u003C\u002Fdesc>\u003Cdefs>\u003Cmarker id=\"crl2-ah\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\">\u003Cpath d=\"M0 0L10 5L0 10z\" class=\"d-head\" \u002F>\u003C\u002Fmarker>\u003C\u002Fdefs>\u003Crect x=\"20\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"79\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Whole document\u003C\u002Ftext>\u003Ctext x=\"79\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">up to 8k tokens\u003C\u002Ftext>\u003Cpath d=\"M138 85H158\" class=\"d-line\" marker-end=\"url(#crl2-ah)\" \u002F>\u003Crect x=\"160\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-accent\" \u002F>\u003Ctext x=\"220\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Transformer\u003C\u002Ftext>\u003Ctext x=\"220\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">one pass\u003C\u002Ftext>\u003Cpath d=\"M279 85H299\" class=\"d-line\" marker-end=\"url(#crl2-ah)\" \u002F>\u003Crect x=\"301\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"360\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Token vectors\u003C\u002Ftext>\u003Ctext x=\"360\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">contextual\u003C\u002Ftext>\u003Cpath d=\"M419 85H439\" class=\"d-line\" marker-end=\"url(#crl2-ah)\" \u002F>\u003Crect x=\"441\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-gold\" \u002F>\u003Ctext x=\"500\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Split spans\u003C\u002Ftext>\u003Ctext x=\"500\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">chunk borders\u003C\u002Ftext>\u003Cpath d=\"M560 85H580\" class=\"d-line\" marker-end=\"url(#crl2-ah)\" \u002F>\u003Crect x=\"582\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-mint\" \u002F>\u003Ctext x=\"641\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Mean pool\u003C\u002Ftext>\u003Ctext x=\"641\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">1 vector\u002Fchunk\u003C\u002Ftext>",[286],"Naive chunking would run the transformer separately on each chunk, so no chunk sees its neighbours.",{"type":127,"content":288},[289],"Jina's original post shows the effect with a Wikipedia text about Berlin: the sentence about the population scored 0.708 cosine similarity to the query \"Berlin\" with naive chunking and 0.825 with late chunking, because \"its\" now carried the city. The paper's benchmarks are more modest. Averaged over three models and four BeIR datasets, late chunking improved nDCG@10 by about 1.8 points with fixed-size chunks, 1.9 with sentence chunks and 1.5 with semantic chunks. The biggest single jump was NFCorpus, from 23.5 to 30.0 with jina-embeddings-v2-small.",{"type":127,"content":291},[292],"The paper is also honest about limits. Gains are largest with small chunks, and on some reading-comprehension sets naive chunking won with large chunks. On synthetic needle and passkey tasks, where the surrounding text is unrelated filler, late chunking did not help. For documents longer than the model's window, the authors split them into overlapping macro chunks and late-chunk each one.",{"type":206,"code":294},"import torch\nfrom transformers import AutoModel, AutoTokenizer\n\n# Any mean-pooling, long-context embedding model whose token outputs you can read.\ntok = AutoTokenizer.from_pretrained(MODEL)\nmodel = AutoModel.from_pretrained(MODEL)\n\ndef late_chunk(text: str, spans: list[tuple[int, int]]) -> list[torch.Tensor]:\n    \"\"\"spans are character ranges from your normal chunker.\"\"\"\n    enc = tok(text, return_offsets_mapping=True, truncation=True,\n              max_length=8192, return_tensors=\"pt\")\n    offsets = enc.pop(\"offset_mapping\")[0].tolist()\n    with torch.no_grad():\n        tokens = model(**enc).last_hidden_state[0]      # one vector per token\n    out = []\n    for start, end in spans:\n        idx = [i for i, (s, e) in enumerate(offsets) if e > s and s >= start and e \u003C= end]\n        v = tokens[idx].mean(dim=0)                     # pool only this chunk's tokens\n        out.append(v \u002F v.norm())\n    return out",{"type":140,"level":141,"id":108,"text":109},{"type":127,"content":297},[298],"The third option packages the idea as a model. Voyage released voyage-context-3 in July 2025 and voyage-context-4 in June 2026: you send all chunks of a document together and get one vector per chunk, each computed with the whole document in view. The API takes a list of documents, each a list of chunks, with limits of 1,000 inputs, 120,000 total tokens and 16,000 chunks per request. Queries go through the same endpoint as single-element lists. voyage-context-4 costs $0.12 per million tokens, can chunk for you, and handles documents beyond 32,000 tokens by splitting them.",{"type":206,"code":300},"import voyageai\n\nvo = voyageai.Client()\n\ndocs = [[\"Chunk 1 of manual A ...\", \"Chunk 2 of manual A ...\"],\n        [\"Chunk 1 of manual B ...\"]]\nres = vo.contextualized_embed(inputs=docs, model=\"voyage-context-4\",\n                              input_type=\"document\")\nchunk_vectors = [r.embeddings for r in res.results]   # one list per document\n\nq = vo.contextualized_embed(inputs=[[\"maximum load of the X200 shelf\"]],\n                            model=\"voyage-context-4\", input_type=\"query\")\nquery_vector = q.results[0].embeddings[0]",{"type":127,"content":302},[303],"Voyage reports that voyage-context-3 beat Anthropic-style contextual retrieval by 6.76% and Jina late chunking by 23.66% on chunk-level retrieval. Treat those as vendor numbers, and note that the voyage-context-3 post itself lists two different sets of percentages in its summary and results sections. The architectural point is solid either way: you get document context without an LLM call per chunk, at the price of a vendor lock on both indexing and queries.",{"type":140,"level":141,"id":111,"text":112},{"type":165,"head":306,"rows":317},[307,309,311,313,315],[308],"Approach",[310],"What changes",[312],"Needs",[314],"Helps BM25",[316],"Main cost",[318,329,337,347],[319,321,323,325,327],[320],"Title and heading prefix",[322],"Chunk text",[324],"Parsed structure",[326],"Yes",[328],"Almost none",[330,331,332,334,335],[12],[322],[333],"LLM call per chunk",[326],[336],"LLM tokens at indexing",[338,339,341,343,345],[13],[340],"Vectors only",[342],"Long-context model, token outputs",[344],"No",[346],"Self-hosted inference",[348,350,351,353,354],[349],"Contextualized embedding API",[340],[352],"One vendor for index and queries",[344],[355],"Per-token API price",{"type":357,"variant":358,"title":359,"body":360},"callout","tip","Try the cheap baseline first",[361],[362,363,367],"Before any of the three, prepend the document title and the heading path to every chunk, for example \"X200 shelf manual > Technical data > Load\". It costs nothing at query time, helps BM25 and vectors alike, and if your parser keeps headings, as discussed in ",{"tag":134,"to":364,"children":365},"\u002Fblog\u002Fpdf-document-parsing-for-rag",[366],"PDF parsing for RAG",", it is a few lines of code. Measure it, then measure contextual retrieval against it, not against bare chunks.",{"type":140,"level":141,"id":114,"text":115},{"type":370,"items":371},"list",[372,377,387,392],[373,376],{"tag":262,"children":374},[375],"Self-contained records."," Product data, FAQ entries and tickets already name their subject. Adding context repeats what is there.",[378,381,382,386],{"tag":262,"children":379},[380],"Small knowledge bases."," Anthropic notes that below about 200,000 tokens, roughly 500 pages, putting everything in the prompt can be simpler than retrieval. I weighed that trade-off in ",{"tag":134,"to":383,"children":384},"\u002Fblog\u002Frag-2026-hybrid-agentic-long-context",[385],"RAG in 2026",".",[388,391],{"tag":262,"children":389},[390],"Unrelated surroundings."," Collections of independent notes in one file give the model nothing useful to propagate, as the needle results in the late chunking paper show.",[393,396],{"tag":262,"children":394},[395],"Fast-changing documents."," If documents change daily, contextual retrieval means re-running the LLM over whole documents daily. Late chunking or a contextualized API costs only an embedding pass.",{"type":140,"level":141,"id":117,"text":118},{"type":370,"ordered":399,"items":400},true,[401,406,411,416],[402,405],{"tag":262,"children":403},[404],"Store context and chunk separately."," Index the combined text, but keep both fields. Anthropic notes that generation may improve if the model can tell the added context apart from the original text, and you need the raw chunk for citations.",[407,410],{"tag":262,"children":408},[409],"Version the context prompt."," The prompt is now part of your index. Changing it means re-contextualizing, so give it a version and store that version with each chunk.",[412,415],{"tag":262,"children":413},[414],"Customize for your domain."," Anthropic suggests domain-specific prompts, for example with a glossary, can beat the generic one. For a B2B catalogue I would tell the model to always name the product family and variant.",[417,420,421,425],{"tag":262,"children":418},[419],"Measure retrieval, not vibes."," Compare recall@20 and nDCG@10 on labelled queries before and after, as in ",{"tag":134,"to":422,"children":423},"\u002Fblog\u002Frag-evaluation-metrics",[424],"Evaluating RAG",". The published gains are averages over other people's data.",{"type":140,"level":141,"id":120,"text":121},{"type":127,"content":428},[429],"My rule of thumb: if you run hybrid search, contextual retrieval is the default, because it is the only option that also fixes keyword matches and it works with any embedding model. If your corpus is large and changes often, or documents must not go to an LLM provider, late chunking with a self-hosted long-context model gives part of the benefit at embedding cost. If you are already on a vendor that offers contextualized embeddings and accept the lock-in, that is the least engineering. In all three cases, start from the title and heading prefix and keep the reranker, which contributed the last step from 49% to 67% in Anthropic's numbers.",{"type":140,"level":141,"id":123,"text":124},{"type":370,"ordered":399,"items":432},[433,437,440,443,446,449,452,455],[434],{"tag":435,"href":39,"children":436},"a",[38],[438],{"tag":435,"href":42,"children":439},[41],[441],{"tag":435,"href":45,"children":442},[44],[444],{"tag":435,"href":48,"children":445},[47],[447],{"tag":435,"href":51,"children":448},[50],[450],{"tag":435,"href":54,"children":451},[53],[453],{"tag":435,"href":57,"children":454},[56],[456],{"tag":435,"href":60,"children":457},[59],[459,532,590,643],{"slug":460,"published":5,"minutes":461,"category":7,"tags":462,"keywords":466,"about":479,"sources":489,"cover":526,"og":527,"expertise":63,"locales":528,"lang":65,"title":529,"description":530,"coverAlt":531},"embedding-model-selection-2026",9,[11,9,463,464,465],"MTEB","Vector search","Model selection",[467,468,469,470,471,472,473,474,475,476,477,478],"best embedding model 2026","how to choose an embedding model","embedding model comparison","MTEB leaderboard explained","RTEB benchmark","open source embedding model","multilingual embedding model","voyage-4 vs openai embeddings","Qwen3-Embedding","EmbeddingGemma","embedding model for RAG","switching embedding models re-embed",[480,483,486],{"name":481,"url":482},"Word embedding","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FWord_embedding",{"name":484,"url":485},"Information retrieval","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FInformation_retrieval",{"name":487,"url":488},"Discounted cumulative gain","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FDiscounted_cumulative_gain",[490,493,496,499,502,505,508,511,514,517,520,523],{"title":491,"url":492},"Enevoldsen et al.: MMTEB, Massive Multilingual Text Embedding Benchmark (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.13595",{"title":494,"url":495},"Hugging Face: Introducing RTEB, a new standard for retrieval evaluation (Oct 2025)","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Frteb",{"title":497,"url":498},"OpenAI: Vector embeddings guide","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fembeddings",{"title":500,"url":501},"OpenAI: API pricing","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fpricing",{"title":503,"url":504},"Voyage AI: The Voyage 4 model family, shared embedding space with MoE architecture (Jan 2026)","https:\u002F\u002Fblog.voyageai.com\u002F2026\u002F01\u002F15\u002Fvoyage-4\u002F",{"title":506,"url":507},"Voyage AI: Text embeddings documentation","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fembeddings",{"title":509,"url":510},"Voyage AI: Pricing","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fpricing",{"title":512,"url":513},"Google: Gemini API embeddings documentation","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fembeddings",{"title":515,"url":516},"Google: Gemini API pricing","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fpricing",{"title":518,"url":519},"Cohere: Embed models documentation","https:\u002F\u002Fdocs.cohere.com\u002Fdocs\u002Fcohere-embed",{"title":521,"url":522},"Qwen: Qwen3-Embedding-8B model card","https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-Embedding-8B",{"title":524,"url":525},"Google Developers Blog: Introducing EmbeddingGemma (Sep 2025)","https:\u002F\u002Fdevelopers.googleblog.com\u002Fen\u002Fintroducing-embeddinggemma\u002F","\u002Fimages\u002Fblog\u002Fembedding-model-selection-2026\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fembedding-model-selection-2026\u002Fog.jpg",[65,66,67],"Choosing an embedding model in 2026: leaderboards, costs and a bake-off on your own data","How to pick an embedding model for RAG in 2026: what MTEB and RTEB tell you, hosted vs open weights, prices, dimensions, and a one-week bake-off.","Three columns of stacked blocks, the middle tallest, each measured by a short bar and a wide bracket.",{"slug":533,"published":5,"minutes":461,"category":7,"tags":534,"keywords":538,"about":551,"sources":559,"cover":584,"og":585,"expertise":63,"locales":586,"lang":65,"title":587,"description":588,"coverAlt":589},"rag-access-control-permissions",[9,535,536,537,464],"Access control","Security","Multi-tenancy",[539,540,541,542,543,544,545,546,547,548,549,550],"rag access control","permission aware rag","document level security rag","rag multi tenancy","vector database access control","pgvector row level security","security trimming azure ai search","openfga rag authorization","qdrant multitenancy","pinecone namespaces multitenancy","owasp llm08 vector and embedding weaknesses","rag data leakage",[552,555,558],{"name":553,"url":554},"Access-control list","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FAccess-control_list",{"name":556,"url":557},"Multitenancy","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FMultitenancy",{"name":28,"url":29},[560,563,566,569,572,575,578,581],{"title":561,"url":562},"OWASP GenAI Security Project: LLM08:2025 Vector and Embedding Weaknesses","https:\u002F\u002Fgenai.owasp.org\u002Fllmrisk\u002Fllm082025-vector-and-embedding-weaknesses\u002F",{"title":564,"url":565},"PostgreSQL documentation: Row Security Policies","https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002Fcurrent\u002Fddl-rowsecurity.html",{"title":567,"url":568},"pgvector README: filtering and iterative index scans","https:\u002F\u002Fgithub.com\u002Fpgvector\u002Fpgvector",{"title":570,"url":571},"Qdrant: Multitenancy and partitions guide","https:\u002F\u002Fqdrant.tech\u002Fdocumentation\u002Fguides\u002Fmultiple-partitions\u002F",{"title":573,"url":574},"Pinecone: Implement multitenancy using namespaces","https:\u002F\u002Fdocs.pinecone.io\u002Fguides\u002Findex-data\u002Fimplement-multitenancy",{"title":576,"url":577},"Microsoft Learn: Document-level access control in Azure AI Search","https:\u002F\u002Flearn.microsoft.com\u002Fen-us\u002Fazure\u002Fsearch\u002Fsearch-document-level-access-overview",{"title":579,"url":580},"OpenFGA: RAG authorization use case","https:\u002F\u002Fopenfga.dev\u002Fdocs\u002Fuse-cases\u002Frag-authorization",{"title":582,"url":583},"Morris et al.: Text Embeddings Reveal (Almost) As Much As Text (EMNLP 2023)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2310.06816","\u002Fimages\u002Fblog\u002Frag-access-control-permissions\u002Fcover.webp","\u002Fimages\u002Fblog\u002Frag-access-control-permissions\u002Fog.jpg",[65,66,67],"Access control for RAG: permission-aware retrieval without leaking documents","How RAG assistants leak documents and how to stop it: ACLs on every chunk, pre-filtering, row-level security in pgvector, permission sync and leak tests.","Seven cards in a row, six with a lock badge, one bare card sending an arrow outward.",{"slug":591,"published":592,"minutes":461,"category":7,"tags":593,"keywords":598,"about":609,"sources":617,"cover":637,"og":638,"expertise":63,"locales":639,"lang":65,"title":640,"description":641,"coverAlt":642},"pdf-document-parsing-for-rag","2026-10-16",[9,594,595,596,597],"PDF parsing","OCR","Docling","Document AI",[599,600,601,602,603,604,605,606,607,608],"pdf parsing for rag","best pdf parser for rag","docling vs marker","marker vs mineru","pdf to markdown for llm","extract tables from pdf for rag","ocr for rag pipeline","olmocr bench","scanned pdf rag","document parsing llm",[610,613,616],{"name":611,"url":612},"PDF","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FPDF",{"name":614,"url":615},"Optical character recognition","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FOptical_character_recognition",{"name":28,"url":29},[618,621,624,627,630,633,636],{"title":619,"url":620},"Docling: README, features and license (GitHub)","https:\u002F\u002Fgithub.com\u002Fdocling-project\u002Fdocling",{"title":622,"url":623},"Auer et al.: Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2501.17887",{"title":625,"url":626},"Datalab: Marker README, olmOCR-bench results and license (GitHub)","https:\u002F\u002Fgithub.com\u002Fdatalab-to\u002Fmarker",{"title":628,"url":629},"OpenDataLab: MinerU README and license (GitHub)","https:\u002F\u002Fgithub.com\u002Fopendatalab\u002FMinerU",{"title":631,"url":632},"Poznanski et al.: olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.18443",{"title":634,"url":635},"Claude Platform Docs: PDF support","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fbuild-with-claude\u002Fpdf-support",{"title":38,"url":39},"\u002Fimages\u002Fblog\u002Fpdf-document-parsing-for-rag\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fpdf-document-parsing-for-rag\u002Fog.jpg",[65,66,67],"PDF parsing for RAG: layout pipelines, OCR and vision models compared","Why PDF parsing decides RAG quality, how Docling, Marker, MinerU and VLM OCR differ, and how to test a parser on your own documents.","Scattered uneven blocks on a sheet becoming an even row of rectangles on a second sheet.",{"slug":644,"published":592,"minutes":645,"category":7,"tags":646,"keywords":650,"about":660,"sources":670,"cover":691,"og":692,"expertise":63,"locales":693,"lang":65,"title":694,"description":695,"coverAlt":696},"multilingual-rag-german-hungarian",7,[9,647,11,648,649],"Multilingual","German","Hungarian",[651,652,653,654,473,655,656,657,658,659],"multilingual rag","german rag embeddings","hungarian nlp search","cross-lingual retrieval","bge-m3 german","german compound words search","hungarian stemming postgres","multilingual-e5-large-instruct","qwen3 embedding multilingual",[661,664,667],{"name":662,"url":663},"Cross-language information retrieval","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FCross-language_information_retrieval",{"name":665,"url":666},"Agglutinative language","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FAgglutinative_language",{"name":668,"url":669},"Compound (linguistics)","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FCompound_(linguistics)",[671,673,676,679,682,685,688],{"title":672,"url":492},"Enevoldsen et al.: MMTEB: Massive Multilingual Text Embedding Benchmark (2025)",{"title":674,"url":675},"Chen et al.: M3-Embedding (BGE-M3): Multi-Linguality, Multi-Functionality, Multi-Granularity (2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2402.03216",{"title":677,"url":678},"Zhang et al.: Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2506.05176",{"title":680,"url":681},"Hugging Face: intfloat\u002Fmultilingual-e5-large-instruct model card","https:\u002F\u002Fhuggingface.co\u002Fintfloat\u002Fmultilingual-e5-large-instruct",{"title":683,"url":684},"Elasticsearch Reference: Hyphenation decompounder token filter","https:\u002F\u002Fwww.elastic.co\u002Fdocs\u002Freference\u002Ftext-analysis\u002Fanalysis-hyp-decomp-tokenfilter",{"title":686,"url":687},"PostgreSQL 18 Documentation: Text search dictionaries","https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002Fcurrent\u002Ftextsearch-dictionaries.html",{"title":689,"url":690},"Snowball: stemming algorithms by language","https:\u002F\u002Fsnowballstem.org\u002Falgorithms\u002F","\u002Fimages\u002Fblog\u002Fmultilingual-rag-german-hungarian\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fmultilingual-rag-german-hungarian\u002Fog.jpg",[65,66,67],"Multilingual RAG for German and Hungarian: embeddings, compounds and cross-lingual search","How to build RAG over German and Hungarian content: multilingual embeddings, compound words, stemming, accents and per-language evals.","Two rows of small squares, each with a dot above, merging into one tall rectangle.",1792222776072]