[{"data":1,"prerenderedAt":759},["ShallowReactive",2],{"blog-table-extraction-rag-en":3},{"slug":4,"published":5,"updated":6,"minutes":7,"category":8,"tags":9,"keywords":15,"about":26,"sources":36,"cover":73,"og":74,"expertise":75,"locales":76,"lang":77,"title":80,"description":81,"coverAlt":82,"metaTitle":83,"takeaways":84,"faq":90,"toc":106,"blocks":134,"others":506},"table-extraction-rag","2026-09-29","2026-10-16",8,"rag",[10,11,12,13,14],"RAG","Tables","Document parsing","Evaluation","Text-to-SQL",[16,17,18,19,20,21,22,23,24,25],"table extraction for RAG","tables in RAG pipelines","pdfplumber vs Camelot","Table Transformer PubTables-1M","Docling table mode TableFormer","Azure Document Intelligence tables","TEDS metric limitations","markdown vs HTML table for LLM","text-to-SQL for table questions","LlamaParse table extraction",[27,30,33],{"name":28,"url":29},"Retrieval-augmented generation","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FRetrieval-augmented_generation",{"name":31,"url":32},"Table (information)","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FTable_%28information%29",{"name":34,"url":35},"PDF","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FPDF",[37,40,43,46,49,52,55,58,61,64,67,70],{"title":38,"url":39},"Factorial study of parsers, chunkers and retrievers (arXiv 2609.31660)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.31660",{"title":41,"url":42},"pdfplumber on GitHub","https:\u002F\u002Fgithub.com\u002Fjsvine\u002Fpdfplumber",{"title":44,"url":45},"Camelot documentation","https:\u002F\u002Fcamelot-py.readthedocs.io\u002Fen\u002Flatest\u002F",{"title":47,"url":48},"Microsoft Table Transformer (archived)","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Ftable-transformer",{"title":50,"url":51},"PubTabNet and TEDS (arXiv 1911.10683)","https:\u002F\u002Farxiv.org\u002Fabs\u002F1911.10683",{"title":53,"url":54},"Semantic evaluation of table extraction (arXiv 2603.18652)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2603.18652",{"title":56,"url":57},"TableFormer (arXiv 2203.01017)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2203.01017",{"title":59,"url":60},"Docling docs: advanced options","https:\u002F\u002Fdocling-project.github.io\u002Fdocling\u002Fusage\u002Fadvanced_options\u002F",{"title":62,"url":63},"Azure Document Intelligence: layout model","https:\u002F\u002Flearn.microsoft.com\u002Fen-us\u002Fazure\u002Fai-services\u002Fdocument-intelligence\u002Fprebuilt\u002Flayout",{"title":65,"url":66},"Table Meets LLM (WSDM 2024)","https:\u002F\u002Fzmy.io\u002Ffiles\u002Fwsdm24-SUC.pdf",{"title":68,"url":69},"TableRAG (arXiv 2410.04739)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.04739",{"title":71,"url":72},"Chain-of-Table (arXiv 2401.04398)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2401.04398","\u002Fimages\u002Fblog\u002Ftable-extraction-rag\u002Fcover.webp","\u002Fimages\u002Fblog\u002Ftable-extraction-rag\u002Fog.jpg","ai-engineer",[77,78,79],"en","de","hu","Table extraction for RAG: extraction layers, serialization and queries that hold","Tables in RAG: why chunking breaks them, pdfplumber, Camelot, Docling and cloud extractors, what TEDS misses, serialization, and routing questions to SQL.","A table on a page becomes separate row strips that each keep a header tab.","Table extraction for RAG · Balázs Csorba",[85,86,87,88,89],"Fixed-size chunking cuts rows away from their headers, so a table question becomes answerable only by luck; a 2026 study found one dense model failing badly on table-derived questions.","Extractors return different shapes: pdfplumber and Camelot read text-based PDFs, Docling’s TableFormer and Table Transformer recognize structure, and Azure Layout and Textract return typed cell grids with headers.","Published parser scores rarely predict your corpus: in a 2026 study TEDS and GriTS correlated with human judgment at r = 0.68 and 0.70, an LLM judge at 0.93.","Keep each table as one unit plus per-row children that repeat the header names, and keep a typed grid next to the text version for citations and SQL.","Route lookup questions to retrieval and aggregation questions such as sums, counts and maxima to SQL or a dataframe, because vector similarity has no notion of sum or max.",[91,94,97,100,103],{"q":92,"a":93},"Why do tables break RAG pipelines?","A table is a two-dimensional structure whose meaning depends on headers. Fixed-size chunking cuts rows away from their headers and flattens numbers into prose, so the retriever finds fragments like \"12.5 | 40 | yes\" with no column names. The model then has to guess what the numbers mean.",{"q":95,"a":96},"Which tool should I use to extract tables from PDFs?","For text-based PDFs, start with pdfplumber or Camelot, which read the PDF’s own text layer. For complex structures, Docling with TableFormer recognizes rows, columns and spans. For scans or mixed documents, cloud layout APIs such as Azure Document Intelligence or Amazon Textract return typed cell grids. Test two of them on your own tables.",{"q":98,"a":99},"Should tables be stored as Markdown or HTML for an LLM?","Markdown is compact and works for simple tables; HTML keeps merged cells and multi-level headers. Research such as Table Meets LLM shows that accuracy changes with format and order, so test both on your own questions. Keep a typed grid alongside whichever text form you index.",{"q":101,"a":102},"How do I answer questions like \"which product has the highest price\" over tables?","Route them away from vector search. Aggregation, counting, ranking and comparison questions need SQL or a dataframe over a table store. Retrieval can find the right table and its schema, but the computation should run as a query, with the usual guardrails for generated SQL.",{"q":104,"a":105},"How do I evaluate table extraction quality?","Build a small judged set of your own tables and score extraction against it, including cell content, not just structure. Published TEDS and GriTS numbers measure benchmark corpora; a 2026 study found them only moderately correlated with human judgment, while an LLM judge correlated much better.",[107,110,113,116,119,122,125,128,131],{"id":108,"title":109},"why-tables-break-chunk-based-rag","Why tables break chunk-based RAG",{"id":111,"title":112},"the-extraction-layers","The extraction layers and what they return",{"id":114,"title":115},"what-the-benchmarks-actually-measure","What the benchmarks actually measure",{"id":117,"title":118},"serialization-choices","Serialization: Markdown, HTML, JSON or prose",{"id":120,"title":121},"chunking-a-table-without-breaking-it","Chunking a table without breaking it",{"id":123,"title":124},"routing-questions-not-documents","Route questions, not just documents",{"id":126,"title":127},"when-not-to-write-a-table-pipeline","When a table pipeline is not worth it",{"id":129,"title":130},"pipeline-checklist","A pipeline checklist",{"id":132,"title":133},"sources","Sources",[135,143,152,155,158,166,172,173,193,220,247,321,345,346,358,365,372,373,380,388,389,392,394,397,398,401,422,426,429,430,437,440,441,460,467,468],{"type":136,"content":137},"paragraph",[138,142],{"tag":139,"children":140},"strong",[141],"Table extraction for RAG"," is the part of document ingestion where most pipelines quietly lose accuracy. A table is a two-dimensional structure whose numbers mean something only together with their row and column headers, and the default RAG recipe of splitting text into fixed-size chunks destroys exactly that relationship. The retriever then returns fragments such as “12.5 | 40 | yes” and the model has to guess which column was which.",{"type":136,"content":144},[145,146,151],"This article treats tables as first-class objects: which extraction layer to use and what it returns, what the benchmarks actually measure, how to serialize and chunk a table, how to route questions that need computation, and when a dedicated table pipeline is not worth it. Whole-document parsing, OCR and layout models are covered in ",{"tag":147,"to":148,"children":149},"link","\u002Fblog\u002Fpdf-document-parsing-for-rag",[150],"PDF and document parsing for RAG","; here a parser is assumed and the focus is the table it found.",{"type":153,"level":154,"id":108,"text":109},"heading",2,{"type":136,"content":156},[157],"Tables break chunk-based RAG because chunking assumes meaning flows along a line of text, and in a table it flows along two axes. A fixed-size window cuts rows away from the header, separates a row from its caption and splits wide rows across chunks. A lookup question such as “what is the maximum load of model X200” can then be answered only if the right row and the header happen to land in the same chunk.",{"type":136,"content":159},[160,161,165],"The effect is measurable. A controlled 2026 factorial study compared three parsers, three chunking strategies and five dense embedding models plus BM25 on regulatory documents, 72,000 result rows in total. It found that parser and chunker choice interact significantly and that one dense model, MPNet-base, showed “a severe failure mode on table-derived questions” (",{"tag":162,"href":39,"children":163},"a",[164],"factorial study","). Text preservation at ingestion was above 98 percent, so the losses came from how tables were split and ranked, not from missing text.",{"type":167,"src":168,"alt":169,"wide":170,"caption":171},"image","\u002Fimages\u002Fblog\u002Ftable-extraction-rag\u002Fdiagram-5.webp","Fixed-size chunks versus one table unit Two columns for the same wide table. Left, fixed-size chunks: chunk 1 holds the header and rows 1 and 2; chunk 2 holds rows 3 to 5 without any header; chunk 3 starts in the middle of row 6. A question about row 4 returns numbers with no column names. Right, one table unit: a parent unit with caption, header and all rows; child units per row that repeat the column names, such as Model: X200; Voltage: 24 V; Max load: 40 kg. The same question finds a row that carries its own meaning.",true,"Fixed-size chunks separate rows from their headers; a table unit with per-row children keeps every row interpretable on its own.",{"type":153,"level":154,"id":111,"text":112},{"type":136,"content":174},[175,176,180,181,184,185,188,189,192],"There are five layers, from cheap and literal to expensive and interpretive, and they return different shapes. PDF geometry is the first: pdfplumber finds explicit lines and lines implied by word alignment, computes their intersections and groups the resulting cells into tables, with strategies such as ",{"tag":177,"children":178},"code",[179],"lines",", ",{"tag":177,"children":182},[183],"text"," and ",{"tag":177,"children":186},[187],"explicit",". Its README is clear that it works best on machine-generated PDFs and does not handle OCR (",{"tag":162,"href":42,"children":190},[191],"pdfplumber",").",{"type":136,"content":194},[195,196,180,199,180,202,184,205,208,209,212,213,216,217,192],"Rule-based parsers are the second layer. Camelot 2.0 offers ",{"tag":177,"children":197},[198],"stream",{"tag":177,"children":200},[201],"lattice",{"tag":177,"children":203},[204],"network",{"tag":177,"children":206},[207],"hybrid"," flavours plus an optional ",{"tag":177,"children":210},[211],"ml"," flavour that runs the Table Transformer family but fills cell text from the PDF’s own text layer, “so it can’t hallucinate or alter a value”; each table comes with a parsing report and a confidence score, and it works only with text-based PDFs (",{"tag":162,"href":45,"children":214},[215],"Camelot","). Learned structure recognition is the third: Microsoft’s Table Transformer, trained on PubTables-1M with 947,642 annotated tables, detects rows, columns and header cells but needs separate text extraction, and its repository was archived read-only in September 2026 (",{"tag":162,"href":48,"children":218},[219],"Table Transformer",{"type":136,"content":221},[222,223,226,227,230,231,234,235,238,239,242,243,192],"Docling packages that third layer with its own model, TableFormer, which predicts structure and takes cell content from the PDF’s text layer; it raised TEDS on PubTabNet from 91 to 98.5 percent on simple tables and from 88.7 to 95 percent on complex ones (",{"tag":162,"href":57,"children":224},[225],"TableFormer","). In Docling you choose ",{"tag":177,"children":228},[229],"TableFormerMode.ACCURATE"," or ",{"tag":177,"children":232},[233],"FAST",", and turn off ",{"tag":177,"children":236},[237],"do_cell_matching"," when columns get merged (",{"tag":162,"href":60,"children":240},[241],"Docling","). Cloud layout APIs are the fourth layer and vision-language parsers such as LlamaParse’s agentic tiers the fifth, priced by credits at 1,000 credits for $1.25 (",{"tag":162,"href":244,"children":245},"https:\u002F\u002Fwww.llamaindex.ai\u002Fpricing",[246],"LlamaIndex",{"type":248,"head":249,"rows":260},"table",[250,252,254,256,258],[251],"Tool",[253],"Method",[255],"Output shape",[257],"Scans",[259],"Cost \u002F license",[261,271,280,291,302,311],[262,263,265,267,269],[191],[264],"PDF geometry",[266],"list of rows of strings",[268],"no",[270],"MIT",[272,274,276,278,279],[273],"Camelot 2.0",[275],"rules, optional ML",[277],"tables with parsing report",[268],[270],[281,283,285,287,289],[282],"Docling (TableFormer)",[284],"learned structure",[286],"cell grid with spans",[288],"with OCR",[290],"MIT, self-hosted",[292,294,296,298,300],[293],"Azure Layout",[295],"cloud layout API",[297],"cells with row, column, kind, spans",[299],"yes",[301],"$10 per 1,000 pages",[303,305,306,308,309],[304],"Textract TABLES",[295],[307],"cell blocks, merged cells, headers",[299],[310],"$15 per 1,000 pages (us-east-1)",[312,314,316,318,319],[313],"LlamaParse",[315],"LLM or VLM parsing",[317],"Markdown, JSON, HTML tables",[299],[320],"credits, from about 1 per page",{"type":136,"content":322},[323,324,327,328,331,332,335,336,339,340,344],"The cloud options return the richest shape. Azure Document Intelligence’s ",{"tag":177,"children":325},[326],"prebuilt-layout"," model gives each table a row and column count and cells with ",{"tag":177,"children":329},[330],"kind"," such as ",{"tag":177,"children":333},[334],"columnHeader",", row and column index, spans and bounding regions, and can output Markdown directly (",{"tag":162,"href":63,"children":337},[338],"Azure","); Textract’s TABLES feature returns merged cells, column headers, titles, footers and summary cells (",{"tag":162,"href":341,"children":342},"https:\u002F\u002Fdocs.aws.amazon.com\u002Ftextract\u002Flatest\u002Fdg\u002Fhow-it-works-tables.html",[343],"AWS","). Prices are as of October 2026, from the providers’ price lists for West Europe and us-east-1 respectively.",{"type":153,"level":154,"id":114,"text":115},{"type":136,"content":347},[348,349,352,353,357],"Most published table numbers use one of two metrics. TEDS, from the PubTabNet paper with its 568,000 table images, turns tables into HTML trees and measures tree edit distance (",{"tag":162,"href":51,"children":350},[351],"PubTabNet","); some leaderboards report TEDS-Struct, which ignores cell text. GriTS evaluates a table directly as a matrix and scores topology, cell location and cell content in one framework (",{"tag":162,"href":354,"children":355},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2203.12555",[356],"GriTS","). Both are useful for comparing models on the same benchmark.",{"type":136,"content":359},[360,361,364],"They predict your corpus less well than the numbers suggest. A 2026 validation study collected more than 1,500 human quality judgments on extracted tables from 21 parsers and found that TEDS correlated with human judgment at r = 0.68 and GriTS at r = 0.70, while an LLM-as-judge pipeline reached r = 0.93; the authors note that rule-based metrics capture structure but “fail to capture semantic equivalence of table content” (",{"tag":162,"href":54,"children":362},[363],"validation study","). The practical conclusion is to build a small judged set of your own tables and score candidates against it.",{"type":366,"variant":367,"title":368,"body":369},"callout","warn","A parser leaderboard is not your corpus",[370],[371],"Datasheets, price lists and regulatory annexes look nothing like the scientific tables in PubTabNet or PubTables-1M. Twenty to fifty of your own tables, checked cell by cell once, will tell you more about which extractor to use than any published TEDS score, and they become the regression test for every later parser or prompt change.",{"type":153,"level":154,"id":117,"text":118},{"type":136,"content":374},[375,376,379],"How a table is written into the index changes what the model can do with it. Markdown is compact and readable but cannot express merged cells or multi-level headers; HTML can, at a higher token cost; JSON records keep types and make each row self-describing. The “Table Meets LLM” study showed that model accuracy on structural tasks such as cell lookup and row retrieval varies with input format, content order, role prompting and partition marks (",{"tag":162,"href":66,"children":377},[378],"Table Meets LLM","), so the choice should be tested, not assumed.",{"type":136,"content":381},[382,383,387],"A fourth option is to rewrite tables as prose. D-RAC, a 2026 pipeline, renders pages, converts them with one multimodal pass into retrieval-oriented Markdown and rewrites tables “as self-contained prose statements” before deterministic chunking (",{"tag":162,"href":384,"children":385},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.24220",[386],"D-RAC","). That works well for small descriptive tables and loses precision for large numeric ones. Whichever text form you index, keep the original typed grid next to it, so answers can cite the exact cell and computation can run on real numbers.",{"type":153,"level":154,"id":120,"text":121},{"type":136,"content":390},[391],"Treat one table as one retrievable parent unit and add one child unit per row that repeats the column names, so a row such as “Model: X200; Voltage: 24 V; Max load: 40 kg” is understandable on its own. Never split the header from the body, attach the caption and the section path, and store the page and table id so the answer can cite them. Large tables can be split into row groups, but each group carries the header again.",{"type":177,"code":393},"# One table -> one parent unit + one child per row that repeats the header names\ndef table_units(table, doc_id, page, caption, section_path):\n    header = [c.strip() for c in table.header]\n    parent = {\n        \"id\": f\"{doc_id}:p{page}:t{table.index}\",\n        \"text\": f\"{section_path} > {caption}\\n\" + table.to_markdown(),\n        \"kind\": \"table\", \"page\": page,\n    }\n    children = []\n    for r, row in enumerate(table.rows):\n        pairs = \"; \".join(f\"{h}: {v}\" for h, v in zip(header, row) if v not in (\"\", None))\n        children.append({\n            \"id\": f\"{parent['id']}:r{r}\",\n            \"text\": f\"{caption}. {pairs}\",          # each row is readable on its own\n            \"kind\": \"table_row\", \"parent\": parent[\"id\"], \"page\": page,\n        })\n    return parent, children   # also write table.rows as typed records to a table store",{"type":136,"content":395},[396],"The parent unit answers questions about the table as a whole, the row children answer lookups, and the typed records in a table store answer computations. On my own Nuxt site, where I run LLM features, the rule that keeps answers trustworthy is the same at a smaller scale: anything that will be cited carries its source location in the metadata from the moment it is indexed.",{"type":153,"level":154,"id":123,"text":124},{"type":136,"content":399},[400],"Lookup questions and computation questions need different machinery. “What is the operating temperature of X200” is a lookup that hybrid retrieval over the row children answers well. “Which model has the highest load capacity under 30 V” or “how many variants support IP67” is an aggregation, and vector similarity has no notion of maximum, count or sum. Those questions should become a query over the typed table store.",{"type":136,"content":402},[403,404,407,408,411,412,416,417,421],"Research supports splitting the work. TableRAG argues that feeding whole tables to a model does not scale and retrieves the schema and relevant cells before generation (",{"tag":162,"href":69,"children":405},[406],"TableRAG","); Chain-of-Table shows that table reasoning works better as a sequence of operations that transform the table than as a single answer (",{"tag":162,"href":72,"children":409},[410],"Chain-of-Table","); and TableBench reports that even GPT-4 scores modestly compared with humans on industrial-style table questions (",{"tag":162,"href":413,"children":414},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2408.09174",[415],"TableBench","). The safety side of generated SQL is covered in ",{"tag":147,"to":418,"children":419},"\u002Fblog\u002Ftext-to-sql-llm-guardrails",[420],"text-to-SQL with LLM guardrails",".",{"type":167,"src":423,"alt":424,"wide":170,"caption":425},"\u002Fimages\u002Fblog\u002Ftable-extraction-rag\u002Fdiagram-26.webp","Table pipeline for RAG Five steps from left to right: a PDF or scan goes into an extraction layer, from PDF geometry through rule-based parsers and learned structure recognition to cloud layout APIs or vision models; the result is a table object with cells, spans, caption and page; a validation gate checks row and column counts and numeric types; the table is stored twice, as text with header context for retrieval and as a typed grid; and questions are routed either to hybrid retrieval for lookups or to SQL for aggregation, answering with a page-level citation.","The table stays a table from extraction to answer: one text form for retrieval, one typed form for computation and citation.",{"type":136,"content":427},[428],"The router itself can be simple. A small classification prompt or a few rules decide whether a question asks for a value, a list, a count, an extreme or a comparison; the last three go to the query path. Retrieval still runs first in both paths, because it finds which table and which columns the question is about. The query path then generates SQL or dataframe code against that schema, executes it in a sandbox, and returns the result together with the rows it used, so the answer can cite them.",{"type":153,"level":154,"id":126,"text":127},{"type":136,"content":431},[432,433,421],"Skip the dedicated pipeline when tables are rare illustrations rather than the data people ask about, when no table schema repeats across documents so a table store would never be queried, or when you have no page-level evaluation and could not see a regression anyway. In those cases, Markdown output from your existing parser, one table per chunk with its caption, and a reranker on top is a reasonable first version. Tool-level trade-offs for general partitioning are in the ",{"tag":147,"to":434,"children":435},"\u002Fblog\u002Funstructured",[436],"Unstructured review",{"type":136,"content":438},[439],"Cost also shapes the choice. As of October 2026 Azure’s layout model lists $10 per 1,000 pages in West Europe and Textract’s TABLES feature $15 per 1,000 pages for the first million pages a month in us-east-1, while Docling or Camelot on your own CPU cost only compute. For a catalog of a few thousand datasheets that is a small one-off bill either way; for millions of pages re-processed on every parser change, self-hosting the extractor starts to matter.",{"type":153,"level":154,"id":129,"text":130},{"type":442,"ordered":170,"items":443},"list",[444,446,448,450,452,454,456,458],[445],"Detect tables and classify each PDF as text-based or scanned.",[447],"Extract text-layer first; use OCR or a cloud layout API only for scans.",[449],"Validate row and column counts, empty cells and numeric types.",[451],"Store a text form and a typed grid for every table.",[453],"Chunk as one parent unit plus row children that repeat headers.",[455],"Route aggregation questions to SQL or a dataframe, lookups to retrieval.",[457],"Cite page and table id in every answer.",[459],"Re-test on a judged set of your own tables after every parser or prompt change.",{"type":136,"content":461},[462,463,421],"If your RAG system has to answer from datasheets, price lists or regulatory tables and you want those answers to hold up, see my ",{"tag":147,"to":464,"children":465},"\u002Fexpertise\u002Fai-engineer",[466],"AI engineering work",{"type":153,"level":154,"id":132,"text":133},{"type":442,"ordered":170,"items":469},[470,473,476,479,482,485,488,491,494,497,500,503],[471],{"tag":162,"href":39,"children":472},[38],[474],{"tag":162,"href":42,"children":475},[41],[477],{"tag":162,"href":45,"children":478},[44],[480],{"tag":162,"href":48,"children":481},[47],[483],{"tag":162,"href":51,"children":484},[50],[486],{"tag":162,"href":54,"children":487},[53],[489],{"tag":162,"href":57,"children":490},[56],[492],{"tag":162,"href":60,"children":493},[59],[495],{"tag":162,"href":63,"children":496},[62],[498],{"tag":162,"href":66,"children":499},[65],[501],{"tag":162,"href":69,"children":502},[68],[504],{"tag":162,"href":72,"children":505},[71],[507,582,640,698],{"slug":508,"published":509,"minutes":510,"category":8,"tags":511,"keywords":516,"about":529,"sources":539,"cover":576,"og":577,"expertise":75,"locales":578,"lang":77,"title":579,"description":580,"coverAlt":581},"embedding-model-selection-2026","2026-10-17",9,[512,10,513,514,515],"Embeddings","MTEB","Vector search","Model selection",[517,518,519,520,521,522,523,524,525,526,527,528],"best embedding model 2026","how to choose an embedding model","embedding model comparison","MTEB leaderboard explained","RTEB benchmark","open source embedding model","multilingual embedding model","voyage-4 vs openai embeddings","Qwen3-Embedding","EmbeddingGemma","embedding model for RAG","switching embedding models re-embed",[530,533,536],{"name":531,"url":532},"Word embedding","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FWord_embedding",{"name":534,"url":535},"Information retrieval","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FInformation_retrieval",{"name":537,"url":538},"Discounted cumulative gain","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FDiscounted_cumulative_gain",[540,543,546,549,552,555,558,561,564,567,570,573],{"title":541,"url":542},"Enevoldsen et al.: MMTEB, Massive Multilingual Text Embedding Benchmark (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.13595",{"title":544,"url":545},"Hugging Face: Introducing RTEB, a new standard for retrieval evaluation (Oct 2025)","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Frteb",{"title":547,"url":548},"OpenAI: Vector embeddings guide","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fembeddings",{"title":550,"url":551},"OpenAI: API pricing","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fpricing",{"title":553,"url":554},"Voyage AI: The Voyage 4 model family, shared embedding space with MoE architecture (Jan 2026)","https:\u002F\u002Fblog.voyageai.com\u002F2026\u002F01\u002F15\u002Fvoyage-4\u002F",{"title":556,"url":557},"Voyage AI: Text embeddings documentation","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fembeddings",{"title":559,"url":560},"Voyage AI: Pricing","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fpricing",{"title":562,"url":563},"Google: Gemini API embeddings documentation","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fembeddings",{"title":565,"url":566},"Google: Gemini API pricing","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fpricing",{"title":568,"url":569},"Cohere: Embed models documentation","https:\u002F\u002Fdocs.cohere.com\u002Fdocs\u002Fcohere-embed",{"title":571,"url":572},"Qwen: Qwen3-Embedding-8B model card","https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-Embedding-8B",{"title":574,"url":575},"Google Developers Blog: Introducing EmbeddingGemma (Sep 2025)","https:\u002F\u002Fdevelopers.googleblog.com\u002Fen\u002Fintroducing-embeddinggemma\u002F","\u002Fimages\u002Fblog\u002Fembedding-model-selection-2026\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fembedding-model-selection-2026\u002Fog.jpg",[77,78,79],"Choosing an embedding model in 2026: leaderboards, costs and a bake-off on your own data","How to pick an embedding model for RAG in 2026: what MTEB and RTEB tell you, hosted vs open weights, prices, dimensions, and a one-week bake-off.","Three columns of stacked blocks, the middle tallest, each measured by a short bar and a wide bracket.",{"slug":583,"published":509,"minutes":584,"category":8,"tags":585,"keywords":589,"about":601,"sources":609,"cover":634,"og":635,"expertise":75,"locales":636,"lang":77,"title":637,"description":638,"coverAlt":639},"contextual-retrieval-late-chunking",10,[10,586,512,587,588],"Chunking","Contextual retrieval","Late chunking",[590,591,592,593,594,595,596,597,598,599,600],"contextual retrieval","anthropic contextual retrieval","late chunking","contextual chunk embeddings","contextual embeddings rag","contextual bm25","chunk context rag","voyage-context-3","voyage-context-4","late chunking vs contextual retrieval","rag chunking context loss",[602,603,606],{"name":28,"url":29},{"name":604,"url":605},"Okapi BM25","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FOkapi_BM25",{"name":607,"url":608},"Sentence embedding","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FSentence_embedding",[610,613,616,619,622,625,628,631],{"title":611,"url":612},"Anthropic: Introducing Contextual Retrieval (Sep 2024)","https:\u002F\u002Fwww.anthropic.com\u002Fnews\u002Fcontextual-retrieval",{"title":614,"url":615},"Günther et al.: Late Chunking, Contextual Chunk Embeddings Using Long-Context Embedding Models (2024, rev. 2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2409.04701",{"title":617,"url":618},"Jina AI: Late Chunking in Long-Context Embedding Models (Aug 2024)","https:\u002F\u002Fjina.ai\u002Fnews\u002Flate-chunking-in-long-context-embedding-models\u002F",{"title":620,"url":621},"Voyage AI: Introducing voyage-context-3 (Jul 2025)","https:\u002F\u002Fblog.voyageai.com\u002F2025\u002F07\u002F23\u002Fvoyage-context-3\u002F",{"title":623,"url":624},"Voyage AI: voyage-context-4 (Jun 2026)","https:\u002F\u002Fblog.voyageai.com\u002F2026\u002F06\u002F29\u002Fvoyage-context-4\u002F",{"title":626,"url":627},"Voyage AI: Contextualized chunk embeddings documentation","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fcontextualized-chunk-embeddings",{"title":629,"url":630},"Anthropic: Claude API pricing","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fabout-claude\u002Fpricing",{"title":632,"url":633},"Qu, Tu and Bao: Is Semantic Chunking Worth the Computational Cost? (2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.13070","\u002Fimages\u002Fblog\u002Fcontextual-retrieval-late-chunking\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fcontextual-retrieval-late-chunking\u002Fog.jpg",[77,78,79],"Contextual retrieval and late chunking: giving RAG chunks back their context","Chunks lose their document. How contextual retrieval, late chunking and contextualized embedding models fix that, what each costs, and when to skip it.","Four loose square pieces gathered by a wide bracket below, each joined upward to a large sheet.",{"slug":641,"published":509,"minutes":510,"category":8,"tags":642,"keywords":646,"about":659,"sources":667,"cover":692,"og":693,"expertise":75,"locales":694,"lang":77,"title":695,"description":696,"coverAlt":697},"rag-access-control-permissions",[10,643,644,645,514],"Access control","Security","Multi-tenancy",[647,648,649,650,651,652,653,654,655,656,657,658],"rag access control","permission aware rag","document level security rag","rag multi tenancy","vector database access control","pgvector row level security","security trimming azure ai search","openfga rag authorization","qdrant multitenancy","pinecone namespaces multitenancy","owasp llm08 vector and embedding weaknesses","rag data leakage",[660,663,666],{"name":661,"url":662},"Access-control list","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FAccess-control_list",{"name":664,"url":665},"Multitenancy","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FMultitenancy",{"name":28,"url":29},[668,671,674,677,680,683,686,689],{"title":669,"url":670},"OWASP GenAI Security Project: LLM08:2025 Vector and Embedding Weaknesses","https:\u002F\u002Fgenai.owasp.org\u002Fllmrisk\u002Fllm082025-vector-and-embedding-weaknesses\u002F",{"title":672,"url":673},"PostgreSQL documentation: Row Security Policies","https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002Fcurrent\u002Fddl-rowsecurity.html",{"title":675,"url":676},"pgvector README: filtering and iterative index scans","https:\u002F\u002Fgithub.com\u002Fpgvector\u002Fpgvector",{"title":678,"url":679},"Qdrant: Multitenancy and partitions guide","https:\u002F\u002Fqdrant.tech\u002Fdocumentation\u002Fguides\u002Fmultiple-partitions\u002F",{"title":681,"url":682},"Pinecone: Implement multitenancy using namespaces","https:\u002F\u002Fdocs.pinecone.io\u002Fguides\u002Findex-data\u002Fimplement-multitenancy",{"title":684,"url":685},"Microsoft Learn: Document-level access control in Azure AI Search","https:\u002F\u002Flearn.microsoft.com\u002Fen-us\u002Fazure\u002Fsearch\u002Fsearch-document-level-access-overview",{"title":687,"url":688},"OpenFGA: RAG authorization use case","https:\u002F\u002Fopenfga.dev\u002Fdocs\u002Fuse-cases\u002Frag-authorization",{"title":690,"url":691},"Morris et al.: Text Embeddings Reveal (Almost) As Much As Text (EMNLP 2023)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2310.06816","\u002Fimages\u002Fblog\u002Frag-access-control-permissions\u002Fcover.webp","\u002Fimages\u002Fblog\u002Frag-access-control-permissions\u002Fog.jpg",[77,78,79],"Access control for RAG: permission-aware retrieval without leaking documents","How RAG assistants leak documents and how to stop it: ACLs on every chunk, pre-filtering, row-level security in pgvector, permission sync and leak tests.","Seven cards in a row, six with a lock badge, one bare card sending an arrow outward.",{"slug":699,"published":509,"minutes":584,"category":8,"tags":700,"keywords":705,"about":717,"sources":725,"cover":752,"og":753,"expertise":754,"locales":755,"lang":77,"title":756,"description":757,"coverAlt":758},"rag-product-catalog-pim-data",[10,701,702,703,704],"PIM","Product data","B2B e-commerce","Metadata filtering",[706,707,708,709,710,711,712,713,714,715,716],"rag product catalog","rag pim data","llm product data","product catalog chatbot","akeneo rag","pimcore ai search","rag structured data","metadata filtering rag","product attributes embeddings","b2b product assistant","etim classification ai",[718,721,722],{"name":719,"url":720},"Product information management","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FProduct_information_management",{"name":28,"url":29},{"name":723,"url":724},"ETIM (standard)","https:\u002F\u002Fwww.etim-international.com\u002Fclassification\u002F",[726,729,732,735,737,740,743,746,749],{"title":727,"url":728},"Akeneo API: Products, product values, metric values and product models","https:\u002F\u002Fapi.akeneo.com\u002Fconcepts\u002Fproducts.html",{"title":730,"url":731},"Akeneo Help Center: What about products with variants?","https:\u002F\u002Fhelp.akeneo.com\u002Fserenity-take-the-power-over-your-products\u002Fserenity-what-about-products-with-variants",{"title":733,"url":734},"Pimcore documentation: Classification Store","https:\u002F\u002Fdocs.pimcore.com\u002Fplatform\u002FPimcore\u002FObjects\u002FObject_Classes\u002FData_Types\u002FClassification_Store\u002F",{"title":736,"url":724},"ETIM International: Classification",{"title":738,"url":739},"ETIM International: ETIM releases and the IXF format","https:\u002F\u002Fetim-international.com\u002F?p=754",{"title":741,"url":742},"Reddy et al.: Shopping Queries Dataset, a large-scale ESCI benchmark for improving product search (2022)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2206.06588",{"title":744,"url":745},"Amazon Science: esci-data repository","https:\u002F\u002Fgithub.com\u002Famazon-science\u002Fesci-data",{"title":747,"url":748},"Qdrant documentation: Filtering","https:\u002F\u002Fqdrant.tech\u002Fdocumentation\u002Fconcepts\u002Ffiltering\u002F",{"title":750,"url":751},"Sui et al.: Table Meets LLM, can large language models understand structured table data? (WSDM 2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.13062","\u002Fimages\u002Fblog\u002Frag-product-catalog-pim-data\u002Fcover.webp","\u002Fimages\u002Fblog\u002Frag-product-catalog-pim-data\u002Fog.jpg","b2b-ecommerce-developer",[77,78,79],"RAG over product catalogues: turning PIM data into answers that hold","Product data is structured, so chunking it like PDFs fails. How to serialize PIM attributes, route spec questions to filters and keep answers exact.","A frame of nine squares with tiny symbols, funnelling down to one card.",1792342719171]