Table extraction for RAG: extraction layers, serialization and queries that hold

Table extraction for RAG is the part of document ingestion where most pipelines quietly lose accuracy. A table is a two-dimensional structure whose numbers mean something only together with their row and column headers, and the default RAG recipe of splitting text into fixed-size chunks destroys exactly that relationship. The retriever then returns fragments such as “12.5 | 40 | yes” and the model has to guess which column was which.

This article treats tables as first-class objects: which extraction layer to use and what it returns, what the benchmarks actually measure, how to serialize and chunk a table, how to route questions that need computation, and when a dedicated table pipeline is not worth it. Whole-document parsing, OCR and layout models are covered in PDF and document parsing for RAG; here a parser is assumed and the focus is the table it found.

Why tables break chunk-based RAG

Tables break chunk-based RAG because chunking assumes meaning flows along a line of text, and in a table it flows along two axes. A fixed-size window cuts rows away from the header, separates a row from its caption and splits wide rows across chunks. A lookup question such as “what is the maximum load of model X200” can then be answered only if the right row and the header happen to land in the same chunk.

The effect is measurable. A controlled 2026 factorial study compared three parsers, three chunking strategies and five dense embedding models plus BM25 on regulatory documents, 72,000 result rows in total. It found that parser and chunker choice interact significantly and that one dense model, MPNet-base, showed “a severe failure mode on table-derived questions” (factorial study). Text preservation at ingestion was above 98 percent, so the losses came from how tables were split and ranked, not from missing text.

The extraction layers and what they return

There are five layers, from cheap and literal to expensive and interpretive, and they return different shapes. PDF geometry is the first: pdfplumber finds explicit lines and lines implied by word alignment, computes their intersections and groups the resulting cells into tables, with strategies such as lines, text and explicit. Its README is clear that it works best on machine-generated PDFs and does not handle OCR (pdfplumber).

Rule-based parsers are the second layer. Camelot 2.0 offers stream, lattice, network and hybrid flavours plus an optional ml flavour that runs the Table Transformer family but fills cell text from the PDF’s own text layer, “so it can’t hallucinate or alter a value”; each table comes with a parsing report and a confidence score, and it works only with text-based PDFs (Camelot). Learned structure recognition is the third: Microsoft’s Table Transformer, trained on PubTables-1M with 947,642 annotated tables, detects rows, columns and header cells but needs separate text extraction, and its repository was archived read-only in September 2026 (Table Transformer).

Docling packages that third layer with its own model, TableFormer, which predicts structure and takes cell content from the PDF’s text layer; it raised TEDS on PubTabNet from 91 to 98.5 percent on simple tables and from 88.7 to 95 percent on complex ones (TableFormer). In Docling you choose TableFormerMode.ACCURATE or FAST, and turn off do_cell_matching when columns get merged (Docling). Cloud layout APIs are the fourth layer and vision-language parsers such as LlamaParse’s agentic tiers the fifth, priced by credits at 1,000 credits for $1.25 (LlamaIndex).

The cloud options return the richest shape. Azure Document Intelligence’s prebuilt-layout model gives each table a row and column count and cells with kind such as columnHeader, row and column index, spans and bounding regions, and can output Markdown directly (Azure); Textract’s TABLES feature returns merged cells, column headers, titles, footers and summary cells (AWS). Prices are as of October 2026, from the providers’ price lists for West Europe and us-east-1 respectively.

What the benchmarks actually measure

Most published table numbers use one of two metrics. TEDS, from the PubTabNet paper with its 568,000 table images, turns tables into HTML trees and measures tree edit distance (PubTabNet); some leaderboards report TEDS-Struct, which ignores cell text. GriTS evaluates a table directly as a matrix and scores topology, cell location and cell content in one framework (GriTS). Both are useful for comparing models on the same benchmark.

They predict your corpus less well than the numbers suggest. A 2026 validation study collected more than 1,500 human quality judgments on extracted tables from 21 parsers and found that TEDS correlated with human judgment at r = 0.68 and GriTS at r = 0.70, while an LLM-as-judge pipeline reached r = 0.93; the authors note that rule-based metrics capture structure but “fail to capture semantic equivalence of table content” (validation study). The practical conclusion is to build a small judged set of your own tables and score candidates against it.

Serialization: Markdown, HTML, JSON or prose

How a table is written into the index changes what the model can do with it. Markdown is compact and readable but cannot express merged cells or multi-level headers; HTML can, at a higher token cost; JSON records keep types and make each row self-describing. The “Table Meets LLM” study showed that model accuracy on structural tasks such as cell lookup and row retrieval varies with input format, content order, role prompting and partition marks (Table Meets LLM), so the choice should be tested, not assumed.

A fourth option is to rewrite tables as prose. D-RAC, a 2026 pipeline, renders pages, converts them with one multimodal pass into retrieval-oriented Markdown and rewrites tables “as self-contained prose statements” before deterministic chunking (D-RAC). That works well for small descriptive tables and loses precision for large numeric ones. Whichever text form you index, keep the original typed grid next to it, so answers can cite the exact cell and computation can run on real numbers.

Chunking a table without breaking it

Treat one table as one retrievable parent unit and add one child unit per row that repeats the column names, so a row such as “Model: X200; Voltage: 24 V; Max load: 40 kg” is understandable on its own. Never split the header from the body, attach the caption and the section path, and store the page and table id so the answer can cite them. Large tables can be split into row groups, but each group carries the header again.

# One table -> one parent unit + one child per row that repeats the header names
def table_units(table, doc_id, page, caption, section_path):
    header = [c.strip() for c in table.header]
    parent = {
        "id": f"{doc_id}:p{page}:t{table.index}",
        "text": f"{section_path} > {caption}\n" + table.to_markdown(),
        "kind": "table", "page": page,
    }
    children = []
    for r, row in enumerate(table.rows):
        pairs = "; ".join(f"{h}: {v}" for h, v in zip(header, row) if v not in ("", None))
        children.append({
            "id": f"{parent['id']}:r{r}",
            "text": f"{caption}. {pairs}",          # each row is readable on its own
            "kind": "table_row", "parent": parent["id"], "page": page,
        })
    return parent, children   # also write table.rows as typed records to a table store

The parent unit answers questions about the table as a whole, the row children answer lookups, and the typed records in a table store answer computations. On my own Nuxt site, where I run LLM features, the rule that keeps answers trustworthy is the same at a smaller scale: anything that will be cited carries its source location in the metadata from the moment it is indexed.

Route questions, not just documents

Lookup questions and computation questions need different machinery. “What is the operating temperature of X200” is a lookup that hybrid retrieval over the row children answers well. “Which model has the highest load capacity under 30 V” or “how many variants support IP67” is an aggregation, and vector similarity has no notion of maximum, count or sum. Those questions should become a query over the typed table store.

Research supports splitting the work. TableRAG argues that feeding whole tables to a model does not scale and retrieves the schema and relevant cells before generation (TableRAG); Chain-of-Table shows that table reasoning works better as a sequence of operations that transform the table than as a single answer (Chain-of-Table); and TableBench reports that even GPT-4 scores modestly compared with humans on industrial-style table questions (TableBench). The safety side of generated SQL is covered in text-to-SQL with LLM guardrails.

The router itself can be simple. A small classification prompt or a few rules decide whether a question asks for a value, a list, a count, an extreme or a comparison; the last three go to the query path. Retrieval still runs first in both paths, because it finds which table and which columns the question is about. The query path then generates SQL or dataframe code against that schema, executes it in a sandbox, and returns the result together with the rows it used, so the answer can cite them.

When a table pipeline is not worth it

Skip the dedicated pipeline when tables are rare illustrations rather than the data people ask about, when no table schema repeats across documents so a table store would never be queried, or when you have no page-level evaluation and could not see a regression anyway. In those cases, Markdown output from your existing parser, one table per chunk with its caption, and a reranker on top is a reasonable first version. Tool-level trade-offs for general partitioning are in the Unstructured review.

Cost also shapes the choice. As of October 2026 Azure’s layout model lists $10 per 1,000 pages in West Europe and Textract’s TABLES feature $15 per 1,000 pages for the first million pages a month in us-east-1, while Docling or Camelot on your own CPU cost only compute. For a catalog of a few thousand datasheets that is a small one-off bill either way; for millions of pages re-processed on every parser change, self-hosting the extractor starts to matter.

A pipeline checklist

1. Detect tables and classify each PDF as text-based or scanned.
2. Extract text-layer first; use OCR or a cloud layout API only for scans.
3. Validate row and column counts, empty cells and numeric types.
4. Store a text form and a typed grid for every table.
5. Chunk as one parent unit plus row children that repeat headers.
6. Route aggregation questions to SQL or a dataframe, lookups to retrieval.
7. Cite page and table id in every answer.
8. Re-test on a judged set of your own tables after every parser or prompt change.

If your RAG system has to answer from datasheets, price lists or regulatory tables and you want those answers to hold up, see my AI engineering work.

Sources

1. Factorial study of parsers, chunkers and retrievers (arXiv 2609.31660)
2. pdfplumber on GitHub
3. Camelot documentation
4. Microsoft Table Transformer (archived)
5. PubTabNet and TEDS (arXiv 1911.10683)
6. Semantic evaluation of table extraction (arXiv 2603.18652)
7. TableFormer (arXiv 2203.01017)
8. Docling docs: advanced options
9. Azure Document Intelligence: layout model
10. Table Meets LLM (WSDM 2024)
11. TableRAG (arXiv 2410.04739)
12. Chain-of-Table (arXiv 2401.04398)