> How coding agents find code: AST chunking with tree-sitter, repo maps, Merkle-tree index sync, code embedding models in 2026, and when grep beats an index.
>
> Web page: https://balazscsorba.com/blog/code-rag-repository-indexing · Language: English · Also available in: [Deutsch](https://balazscsorba.com/de/blog/code-rag-repository-indexing.md) · [Magyar](https://balazscsorba.com/hu/blog/code-rag-repository-indexing.md)
> Author: Balázs Csorba · Published: 2026-10-01 · Keywords: RAG over code, AST chunking, tree-sitter code chunking, repository map, code embeddings, semantic code search, codebase indexing, repo map tokens, incremental code index, CoIR benchmark

[Blog](https://balazscsorba.com/blog)/Retrieval & search

# RAG over code: AST chunking, repo maps and code embeddings that fit an agent

How coding agents find code: AST chunking with tree-sitter, repo maps, Merkle-tree index sync, code embedding models in 2026, and when grep beats an index.

[Balázs Csorba](https://balazscsorba.com/about)·October 1, 2026·8 min read

-   RAG
-   Code search
-   Tree-sitter
-   Embeddings
-   Coding agents

![A branching code tree feeds a row of index cards.](https://balazscsorba.com/images/blog/code-rag-repository-indexing/cover.webp?v=ba771e90f4)

## Key takeaways

-   Line-based chunking splits functions and separates imports from the code that needs them; the cAST paper measured +4.3 Recall@5 on RepoEval and +2.67 Pass@1 on SWE-bench from AST-based chunking.
-   Tree-sitter is the common parser: chunk by function, class or method, merge small siblings, split oversized nodes, and store path, line range, symbol name and signature with each chunk.
-   A repo map needs no embeddings: Aider ranks tree-sitter symbols over a file dependency graph and fits signatures into a 1k-token default budget.
-   Index freshness is the hard part: Cursor syncs only changed branches of a Merkle tree, Kilo Code uses file watchers and hash caches, and CocoIndex re-embeds only changed files.
-   Grep and glob never go stale and cost tokens per search; an index costs storage, embedding spend and freshness work. Many teams need both: grep for exact names, an index for meaning.

On this page

1.  [Why line-based chunking fails on code](https://balazscsorba.com/#why-line-based-chunking-fails-on-code)
2.  [AST chunking with tree-sitter](https://balazscsorba.com/#ast-chunking-with-tree-sitter)
3.  [Repo maps: symbol context without embeddings](https://balazscsorba.com/#repo-maps-symbol-context-without-embeddings)
4.  [How commercial agents index code](https://balazscsorba.com/#how-commercial-agents-index-code)
5.  [Code embedding models in 2026](https://balazscsorba.com/#code-embedding-models-in-2026)
6.  [Measuring code retrieval and keeping it fresh](https://balazscsorba.com/#measuring-code-retrieval-and-keeping-it-fresh)
7.  [When agentic grep beats an index, and a checklist](https://balazscsorba.com/#when-agentic-grep-beats-an-index)
8.  [Sources](https://balazscsorba.com/#sources)

Listen to this article

0:000:00

**RAG over code** is retrieval-augmented generation where the corpus is a repository, and it fails in different ways than RAG over documents. Code has hard boundaries (functions, classes, imports), exact identifiers that matter more than paraphrase, and a call graph that connects files. Chunk it like prose and the model gets half a function and no imports; index it once and forget it, and the agent confidently reads code that was deleted yesterday.

This article covers structural chunking with tree-sitter, repo maps that need no embeddings, how Cursor and Kilo Code keep an index fresh, the code embedding models available as of October 2026, how to measure retrieval on your own repository, and when grep beats any index. Deciding what goes into an agent’s context in the first place is covered in [the AGENTS.md, skills, MCP and CLI decision matrix](https://balazscsorba.com/blog/agents-md-skills-mcp-cli-decision-matrix); this one is about how the code gets found.

## Why line-based chunking fails on code

Line-based chunking fails because code boundaries do not fall on fixed line counts. A 40-line window splits a function in the middle, leaves a decorator in one chunk and its class in the next, and separates imports from the code that uses them. The retriever then ranks fragments that no longer mean anything on their own.

The cAST paper measured the effect. It describes line-based heuristics as ones that “often break semantic structures, splitting functions or merging unrelated code,” and replaces them with recursive chunking along the abstract syntax tree, merging small sibling nodes until a size limit. That change improved Recall@5 by 4.3 points on RepoEval retrieval and Pass@1 by 2.67 points on SWE-bench generation ([cAST](https://arxiv.org/abs/2506.15655)). The gain is modest in absolute terms, but it comes from a parser, not from a bigger model.

![Line windows versus AST chunks Two columns for the same source file. Column A, fixed line windows: window 1 holds the imports and the first half of parse(); window 2 holds the rest of parse() and a decorator cut off from its class; window 3 holds the class body without its decorator. Column B, AST nodes: a module chunk with imports and file-level constants; a function chunk with the whole parse() body plus path, line range and signature; a class chunk for Order with its decorator kept. A footer notes the cAST results: +4.3 Recall@5 on RepoEval and +2.67 Pass@1 on SWE-bench.](https://balazscsorba.com/images/blog/code-rag-repository-indexing/diagram-5.webp?v=d3e781821a)

Fixed line windows cut definitions apart; AST chunks keep each function and class whole and carry their location and signature.

Code retrieval is also not text retrieval for a second reason: queries mix natural language with exact identifiers. A question like “where do we retry the ERP call” needs semantic matching, while “who calls `createSalesOrder`” needs an exact symbol lookup. A good setup serves both, which is why most of the systems below combine structure with search.

## AST chunking with tree-sitter

Tree-sitter is the common parser across these systems: it parses a file into a concrete syntax tree, and query files such as `tags.scm` define what counts as a definition or a reference for each language ([Aider](https://aider.chat/2023/10/22/repomap.html)). A chunker walks the top-level nodes, emits functions, classes and methods as chunks, merges small neighbors so a file of short helpers does not become fifty tiny chunks, and splits oversized nodes by recursing into their children.

```
# Sketch: AST chunks with sibling merging, the idea behind cAST
from tree_sitter import Language, Parser
import tree_sitter_python as tspython

parser = Parser(Language(tspython.language()))
MAX_CHARS = 1000  # Kilo Code documents 100 to 1,000 characters per block

def chunks(src: bytes, path: str):
    buf = []
    for node in parser.parse(src).root_node.children:
        text = src[node.start_byte:node.end_byte].decode()
        if len(text) > MAX_CHARS and node.child_count:
            yield from split_children(node, src, path)   # oversized: recurse
        elif sum(len(t) for t, _ in buf) + len(text) <= MAX_CHARS:
            buf.append((text, node))                     # merge small siblings
        else:
            yield emit(buf, path)                        # path, lines, symbol, signature
            buf = [(text, node)]
    if buf:
        yield emit(buf, path)
```

Kilo Code, from the Cline and Roo lineage, documents concrete settings for this: tree-sitter parsing into semantic blocks of 100 to 1,000 characters, large functions split, and a line-based fallback for files the parser does not understand ([Kilo Code](https://kilo.ai/docs/customize/context/codebase-indexing)). Whatever the size limits, store metadata with every chunk: the file path, the line range, the symbol name and the signature. The agent needs the path and lines to open the real file, and the signature makes the chunk readable in a result list without its body.

## Repo maps: symbol context without embeddings

A repo map gives the model the shape of a repository without any vector index. Aider builds one with tree-sitter: the key classes and functions with their types and call signatures. It selects what fits a token budget with a graph-ranking algorithm over a graph where each file is a node and dependencies are edges. The `--map-tokens` setting defaults to 1k tokens, and the map expands when no files have been added to the chat yet ([Aider docs](https://aider.chat/docs/repomap.html)).

The map holds signatures, not bodies, which is why it is cheap. Aider’s design note from 2023 explains that tree-sitter replaced ctags because it shows full signatures straight from the source and ships language support as installable packages ([Aider blog](https://aider.chat/2023/10/22/repomap.html)). The retrieval pattern is marks first: the model reads the map, picks the files it needs, and only then reads them in full. For many tasks in a repository you already know, that is enough, and nothing ever has to be re-embedded.

Language servers go one step further. Serena is an MCP toolkit for symbol-level retrieval and editing whose default backend is the Language Server Protocol, covering more than 40 languages ([Serena](https://github.com/oraios/serena)). Instead of searching text, the agent asks for a symbol’s definition or references and gets exact answers. How much that saves in practice is covered in [the token-saving tools for coding agents](https://balazscsorba.com/blog/token-saving-tools-coding-agents-top-20).

## How commercial agents index code

Cursor indexes with a Merkle tree. Each file gets a SHA-256 hash and each folder a hash of its children, so after an edit only the changed branches diverge and need to be synced. In a 50,000-file workspace, the filenames and hashes alone add up to roughly 3.2 MB. Changed files are split into syntactic chunks, embeddings are cached by chunk content, and a similarity hash finds an existing index to reuse, with content proofs so a client only gets results for files it can actually read ([Cursor](https://cursor.com/blog/secure-codebase-indexing)).

The vector layer behind it is turbopuffer, which describes Cursor’s retrieval as more than a trillion vectors with one namespace per codebase instance, embedded with Cursor’s own model. Copying an existing namespace for a new checkout cut the median time to first query from 7.87 seconds to 525 milliseconds, and the p99 from 4.03 hours to 21 seconds ([turbopuffer](https://turbopuffer.com/customers/cursor)). That is the onboarding problem every code index has: the first index of a large repository is the slow part.

Kilo Code ships the same pattern for a local setup: LanceDB by default or Qdrant as an external store, embedding providers including Ollama for a fully local stack, file watching plus hash-based caching for incremental updates, and handling for branch switches. Its defaults include a minimum search score of 0.4 and up to 50 results ([Kilo Code](https://kilo.ai/docs/customize/context/codebase-indexing)). Cline itself, by contrast, currently documents built-in tools such as `read_files` and `search_codebase` and no embedding index ([Cline docs](https://docs.cline.bot/tools-reference/all-cline-tools.md)).

![Indexed retrieval path for code Five steps from left to right: the repository tracked by Merkle hashes; tree-sitter turns changed files into AST chunks; chunks are embedded, with embeddings cached by content hash; vectors go into a store with one namespace per codebase; search returns chunks, and the agent then reads the files.](https://balazscsorba.com/images/blog/code-rag-repository-indexing/diagram-19.webp?v=9eee616d4b)

The indexed path pays once for structure and embeddings; the agentic path (grep, glob, read, repeat) skips every box here and pays in tokens per search instead.

System

Chunking

Retrieval primitive

Storage

Freshness

Aider repo map

tree-sitter tags, signatures only

graph ranking over file dependencies

none; map goes into the prompt

rebuilt per request, 1k-token default

Cursor

syntactic chunks

semantic search over embeddings

turbopuffer, namespace per codebase

Merkle tree, only changed branches synced

Kilo Code

tree-sitter blocks, 100–1,000 chars

vector search, min score 0.4

LanceDB or Qdrant

file watcher plus hash cache

Serena

none; symbols from language servers

LSP definition and reference lookup

language server state

live

Claude Code

none

grep (ripgrep) and glob, just in time

none

never stale; tokens per search

## Code embedding models in 2026

As of October 2026, there are three practical routes. Hosted and code-tuned: Voyage lists `voyage-code-4` as its current code model at $0.12 per million tokens with 200 million free tokens, while `voyage-code-3` has moved to the older models at $0.18 ([Voyage](https://docs.voyageai.com/docs/pricing)). Self-hosted: Qwen3-Embedding comes in 0.6B, 4B and 8B sizes with a 32k context, Matryoshka dimensions and support for more than 100 languages including programming languages ([Qwen](https://qwenlm.github.io/blog/qwen3-embedding/)).

The third route is a general multimodal or long-context model. jina-embeddings-v4 can emit either a single 2048-dimensional vector or token-level vectors for late interaction, reports 71.59 on the CoIR code benchmark, and carries the Qwen Research License, which matters for commercial use ([Jina](https://jina.ai/models/jina-embeddings-v4/)). Cohere Embed v4 offers a 128k context and Matryoshka output from 256 to 1536 dimensions at $0.12 per million tokens ([Cohere](https://docs.cohere.com/changelog/embed-multimodal-v4)). Which model to pick is a general question, covered in [embedding model selection](https://balazscsorba.com/blog/embedding-model-selection-2026); for code, the long context matters because whole files and long functions fit.

For a repository of a few hundred thousand lines, the embedding bill is small at these prices; the real costs are re-embedding after churn and the engineering to keep the index correct. Code-tuning pays most when queries are in natural language and the code uses domain names the general model has never seen.

## Measuring code retrieval and keeping it fresh

Measure on your own repository, with public benchmarks as a sanity check. CoIR, published at ACL 2025, combines ten datasets across eight retrieval tasks and seven domains, and its authors found that even state-of-the-art models had significant difficulties with code retrieval ([CoIR](https://arxiv.org/abs/2407.02883)). Your queries, identifiers and conventions are not in it, so build a small set yourself.

1.  Collect 30 to 50 real questions from tickets and code review, each with the file and symbol that answers it.
2.  Mix natural-language questions with exact-name lookups, since they stress different retrievers.
3.  Score recall at 5 and 10 for each chunking and model variant, and keep the set in the repository.
4.  Re-run it after model changes and large refactors, not only once.

Freshness needs as much design as quality. The working patterns are file watchers with content-hash caches (Kilo Code), Merkle trees that sync only changed branches (Cursor), and memoized pipelines: CocoIndex keys its cache per file and per processing step, so a commit that touches a few files re-embeds only those, deleted files remove their rows, and the index runs either as a catch-up pass in CI or in live mode watching the working tree with about one second of latency ([CocoIndex](https://cocoindex.io/blogs/index-codebase-v1)). Deletions are the case most setups get wrong, which is how an agent ends up citing removed code. Late chunking and contextual retrieval add context to chunks on top of this, see [contextual retrieval and late chunking](https://balazscsorba.com/blog/contextual-retrieval-late-chunking).

**Leaderboards are not your repository**

Benchmark scores for code embeddings change from month to month, and none of them contain your naming conventions. Treat a CoIR number as a filter for the shortlist and let your own question set decide.

## When agentic grep beats an index, and a checklist

Grep beats an index when the repository is small enough to search quickly, changes constantly, or cannot leave the machine. Anthropic describes Claude Code as a hybrid: CLAUDE.md is loaded up front, and “primitives like glob and grep allow it to navigate its environment and retrieve files just-in-time, effectively bypassing the issues of stale indexing and complex syntax trees.” The same post notes that runtime exploration is slower than pre-computed retrieval ([Anthropic](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents)). Claude Code’s Grep tool is built on ripgrep, alongside Glob ([Claude Code docs](https://code.claude.com/docs/en/tools-reference)).

I run Claude Code, opencode and Codex side by side, and the trade-off is visible in daily use: grep-first agents spend tokens on several search-and-read rounds in an unfamiliar area, and never return code that no longer exists. The cost of an index is the opposite: storage, embedding spend and the freshness machinery above, in exchange for fewer rounds on questions phrased by meaning.

-   **Repository size**: below a size where ripgrep answers instantly, start with grep and a repo map.
-   **Churn**: high churn favors grep or LSP; an index needs incremental updates that handle deletions.
-   **Monorepo**: many teams and unfamiliar areas favor an index, ideally shared and reused across checkouts.
-   **Compliance**: if code cannot leave the machine, use local embedders and a local store, or skip the index.
-   **Hybrid by default**: grep for exact names, LSP for definitions and references, an index for meaning.

If you are building retrieval for a coding agent or an internal code assistant, my [AI engineering work](https://balazscsorba.com/expertise/ai-engineer) covers this kind of setup, from chunking to evaluation.

## Sources

1.  [cAST: AST-based chunking for code RAG (arXiv 2506.15655)](https://arxiv.org/abs/2506.15655)
2.  [Aider docs: repository map](https://aider.chat/docs/repomap.html)
3.  [Aider blog: building a better repository map with tree-sitter (2023)](https://aider.chat/2023/10/22/repomap.html)
4.  [Anthropic Engineering: effective context engineering for AI agents (Sep 2025)](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents)
5.  [Cursor blog: securely indexing large codebases (Jan 2026)](https://cursor.com/blog/secure-codebase-indexing)
6.  [turbopuffer: Cursor customer story](https://turbopuffer.com/customers/cursor)
7.  [Kilo Code docs: codebase indexing](https://kilo.ai/docs/customize/context/codebase-indexing)
8.  [Serena on GitHub](https://github.com/oraios/serena)
9.  [Voyage AI docs: pricing](https://docs.voyageai.com/docs/pricing)
10.  [Qwen blog: Qwen3-Embedding (Jun 2025)](https://qwenlm.github.io/blog/qwen3-embedding/)
11.  [CoIR: a benchmark for code information retrieval (arXiv 2407.02883)](https://arxiv.org/abs/2407.02883)
12.  [CocoIndex blog: indexing a codebase](https://cocoindex.io/blogs/index-codebase-v1)

## Frequently asked questions

What is RAG over code?

Retrieval-augmented generation where the corpus is a code repository. Instead of splitting text into paragraphs, the system splits code along syntax, usually functions and classes, indexes those chunks with embeddings or symbol maps, and gives the model the most relevant pieces for a task.

Why use AST chunking instead of fixed line windows?

Fixed windows cut functions in half and separate decorators and imports from the code they belong to. Chunking along the abstract syntax tree keeps whole definitions together. The cAST paper reports +4.3 points Recall@5 on RepoEval and +2.67 points Pass@1 on SWE-bench from recursive AST chunking with sibling merging.

What is a repo map?

A compact summary of a repository’s classes and functions with their signatures, built with tree-sitter. Aider ranks symbols over a graph of file dependencies and includes only what fits a token budget, 1k tokens by default, so the model sees structure without file bodies.

Which embedding model should I use for code in 2026?

As of October 2026, code-tuned options include voyage-code-4 at $0.12 per million tokens and Qwen3-Embedding in 0.6B, 4B and 8B sizes for self-hosting; jina-embeddings-v4 reports 71.59 on CoIR under a research license. Evaluate two or three candidates on queries from your own repository before choosing.

Do coding agents need a vector index at all?

Not always. Claude Code navigates with grep and glob just in time, which avoids stale indexes and syntax-tree complexity but is slower than pre-computed retrieval. An index pays off in large or unfamiliar repositories and for questions phrased by meaning rather than by name.

Written by Balázs Csorba

Senior fullstack & AI engineer in Styria, Austria – 10+ years of Vue, Nuxt, Node.js and PHP, now building tooling for AI agents.

[AI engineering & MCP servers →](https://balazscsorba.com/expertise/ai-engineer)[About me →](https://balazscsorba.com/about)

## More articles

-   [Choosing an embedding model in 2026: leaderboards, costs and a bake-off on your own data](https://balazscsorba.com/blog/embedding-model-selection-2026)
-   [Contextual retrieval and late chunking: giving RAG chunks back their context](https://balazscsorba.com/blog/contextual-retrieval-late-chunking)
-   [Access control for RAG: permission-aware retrieval without leaking documents](https://balazscsorba.com/blog/rag-access-control-permissions)
-   [RAG over product catalogues: turning PIM data into answers that hold](https://balazscsorba.com/blog/rag-product-catalog-pim-data)

## Sounds like what you need?

Tell me about your project or role – I’d love to hear from you.

[Get in touch](mailto:contact@balazscsorba.com) [Connect on LinkedIn](https://www.linkedin.com/in/balazs-csorba)