Context rot is the decline in an LLM's accuracy as its input gets longer, long before the context window is full. Studies such as Lost in the Middle, NoLiMa and Chroma's Context Rot report show that models use long inputs unevenly and become less reliable as irrelevant or distracting content accumulates.

What is compaction in AI agents?

Compaction replaces the older part of a conversation with a model-written summary and continues from it, so a long task fits in a smaller, more focused context. The Claude API offers it at a token threshold or on demand, and OpenAI's Responses API offers it through a compaction threshold or a standalone compact endpoint.

Is compaction enough to keep a long-running agent on track?

Usually not on its own. Anthropic reports that even with compaction, agents tended to attempt too much at once or declare work finished early. Persisting a plan, a progress log, a feature list and git commits, and reading them at the start of each session, made long runs coherent.

What is the difference between context editing and compaction?

Context editing removes specific content by rule, such as old tool results or thinking blocks, without summarising anything. Compaction summarises older turns into a shorter block. Clearing is cheaper and safer; compaction keeps more of the conversation's meaning but loses detail the summary does not capture.

Should I use sub-agents to avoid context rot?

For read-only exploration, yes: a sub-agent can search in its own window and return a summary of about 1,000 to 2,000 tokens. For work that writes or changes things, keep one agent with full context, because sub-agents that cannot see each other's decisions tend to produce conflicting results.

Context rot and compaction in long agent runs

Context rot is the drop in an LLM's accuracy as its input gets longer, well before the context window is full. A 1M-token window does not mean the model uses the millionth token as well as the hundredth. For a chat that hardly matters. For an agent that runs for hours and appends every file it reads and every command it runs, it is the main reason quality falls off late in a task.

This article collects the evidence, explains why agents are hit hardest, and walks through the four levers that work in practice: clearing old tool results, compaction, notes and memory outside the window, and sub-agents with their own context. It includes the current Claude and OpenAI API mechanics for each, and the harness patterns that keep a long run coherent.

What the research shows

Three lines of work point the same way. None of them uses agent workloads, which, if anything, means real agents fare worse:

Study	Setup	Finding
Lost in the Middle (Liu et al., 2023)	Multi-document QA and key-value retrieval with the relevant item at different positions	Accuracy is highest when the relevant information is at the start or end, and drops significantly in the middle, even for long-context models
NoLiMa (Modarressi et al., ICML 2025)	Needle-in-a-haystack where question and needle share minimal wording; 13 models claiming at least 128K context	At 32K tokens, 11 of 13 models fell below 50% of their short-context baseline; GPT-4o dropped from 99.3% to 69.7%
Context Rot (Chroma, July 2025)	18 models including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3 on controlled simple tasks	Performance grows increasingly unreliable as input grows; on LongMemEval every model did significantly better with focused ~300-token prompts than with full ~113K-token prompts

Chroma's report adds two details that matter for agents. Distractors, content that looks relevant but is not, hurt more as context grows, and the Claude models in the study tended to abstain under that ambiguity rather than guess. And, counter-intuitively, models did better on a shuffled haystack than on a coherent one. The conclusion of the report is the useful one: whether the relevant information is in the context matters less than how it is presented.

Why agents are hit hardest

The agent loop appends every tool call and every tool result to the history and sends all of it back on the next turn. After two hours of work the context is mostly stale file reads, old test output and superseded plans: exactly the distractor-heavy, low-signal input the studies above punish. Anthropic's context engineering guide frames it as an "attention budget": a transformer relates every token to every other token, n² pairs for n tokens, so each added token thins the attention left for the ones that matter. The guide's working rule is to find "the smallest possible set of high-signal tokens" for the next step.

The second problem is cost. Every token in the history is billed again on every turn; caching softens that (see prompt caching and routing), but a context that only grows makes every later step slower and more expensive as well as less accurate.

Four levers: clear, compact, offload, isolate

Lever	What it does	What you lose	Reach for it when
tagchildren	Drops old tool results (and optionally inputs or thinking) by rule	The raw output, which the agent can usually re-fetch	Tool output dominates the context; first and cheapest step
tagchildren	Replaces older turns with a model-written summary	Detail the summary did not keep	Long back-and-forth that must stay one conversation
tagchildren	Agent writes notes, plans and progress to files or a memory store	Nothing, if the agent reads them back	Multi-session work with clear milestones
tagchildren	A sub-agent explores in its own window and returns a short summary	The sub-agent's reasoning; coordination gets harder	Broad search or research that would flood the main window

Illustrative, not measured: clearing slows growth, compaction keeps the context in a band. The point of both is to stay where the model is accurate, not just under the limit.

Clearing old tool results

Anthropic calls tool result clearing "one of the safest lightest touch forms of compaction": once the agent has read a file and acted on it, the full file contents rarely need to stay. On the Claude API this is context editing (beta header context-management-2025-06-27). The clear_tool_uses_20250919 strategy activates above a trigger (default 100,000 input tokens), keeps the most recent tool uses (default 3) and replaces older results with a placeholder:

response = client.beta.messages.create(
    model="claude-opus-5-5",
    max_tokens=16000,
    betas=["context-management-2025-06-27"],
    tools=tools,
    messages=messages,
    context_management={"edits": [{
        "type": "clear_tool_uses_20250919",
        "trigger": {"type": "input_tokens", "value": 30000},
        "keep": {"type": "tool_uses", "value": 3},
        "clear_at_least": {"type": "input_tokens", "value": 5000},
        "exclude_tools": ["web_search"],   # results you never want dropped
    }]},
)

Two settings deserve attention. exclude_tools protects results the agent cannot cheaply recreate. clear_at_least exists because clearing invalidates the cached prompt prefix at the point of the edit; if a clearing pass only frees a few hundred tokens, the cache rewrite costs more than it saves. A separate clear_thinking_20251015 strategy does the same for old thinking blocks.

You can do the same in your own harness without any beta: keep the last few tool results verbatim and replace older ones with a one-line stub such as "read src/order.ts (412 lines), see notes". The stub keeps the trajectory legible while dropping the bulk.

Compaction: summarise and continue

Compaction replaces older turns with a summary and continues from it. Anthropic describes how Claude Code does it: the model summarises the history, keeping architectural decisions, unresolved bugs and implementation details while discarding redundant tool outputs. Both big APIs now offer it server-side:

Claude, at a token threshold (beta compact-2026-01-12): add compact_20260112 to context_management.edits. The default trigger is 150,000 input tokens, with a minimum of 50,000. instructions replaces the default summarisation prompt entirely, and pause_after_compaction stops after the summary so you can re-insert recent turns verbatim.
Claude, on demand (beta compact-2026-09-04): send the conversation with "compaction": {"type": "summarize"}; the response is a single signed compaction block with stop_reason "compaction", which goes first in your next request in place of the summarised messages. Your application decides when.
OpenAI Responses API: set context_management with compact_threshold on responses.create, or call the standalone compact endpoint. The returned compaction item is encrypted and "not intended to be human-interpretable"; with store=false the flow is ZDR-friendly.

response = client.beta.messages.create(
    model="claude-opus-5-5",
    max_tokens=16000,
    betas=["compact-2026-01-12"],
    tools=tools,
    messages=messages,
    context_management={"edits": [{
        "type": "compact_20260112",
        "trigger": {"type": "input_tokens", "value": 120000},
        "instructions": (
            "Summarise for continuing this task in a new context window. Keep: the goal, "
            "constraints the user stated, decisions made and why, files changed, failing "
            "tests and their errors, open questions, the next step. Drop raw tool output."
        ),
    }]},
)
messages.append({"role": "assistant", "content": response.content})   # keep the compaction block

A summary is lossy by design. What the default prompt drops is usually what bites later: the user's constraint from turn three, the reason an approach was abandoned. Write your own instructions for your domain, keep the last few turns verbatim, and test it: run tasks that need a fact from before the compaction and check the agent still has it.

Notes and memory outside the window

Anthropic's November 2025 harness article is blunt: "compaction isn't sufficient". Even a frontier model looping across context windows tended to try to one-shot an app or to declare the project done early. What fixed it was state on disk. An initializer session writes an init.sh, a claude-progress.txt log and an initial git commit; for their example app it also wrote a feature list of more than 200 requirements, all marked failing. Every later session reads the progress file and git history, makes incremental progress on one feature and leaves structured updates.

This is the offload lever, and it is the one I rely on most for coding agents: a plan file, a progress log and small commits are cheap, survive any compaction or crash, and are readable by the human reviewing the work. For agents that are not file-based, Claude's memory tool (memory_20250818) gives the same pattern as a tool: the model reads and writes files under /memories, and your application executes those operations against storage you control. Because it is your storage, the handler must reject paths outside /memories. What to store, and how to keep it from being poisoned, is covered in designing memory for AI agents.

Sub-agents: isolation with a price

A sub-agent gets a task, its own clean context window and its own tools. It may burn tens of thousands of tokens exploring, but returns, in Anthropic's words, "a condensed, distilled summary of its work (often 1,000-2,000 tokens)". The main agent's window only ever sees the summary. That is the strongest lever against rot for search-heavy work, such as "find every place that computes VAT" across a large codebase.

The price is coordination. Cognition's "Don't Build Multi-Agents" argues for sharing full agent traces rather than individual messages, because actions carry implicit decisions and sub-agents that cannot see each other's decisions produce conflicting results. My rule: sub-agents for read-only exploration and research that returns facts; one agent with full context for anything that writes. When multi-agent systems are worth it goes deeper.

Load less in the first place

The cheapest token to manage is the one never added. Anthropic describes "just in time" context: the agent keeps lightweight identifiers (file paths, stored queries, links) and loads data only when needed, using tools like head, tail and targeted queries instead of reading whole objects. Tool design does the rest:

Paginate and truncate tool output by default, with a clear note on how to get more.
Return summaries plus handles, not raw dumps: "143 matching orders, first 10 below, query id q_17".
Keep stable instructions short and up front, and put the current goal and constraints near the end of the context, where the Lost in the Middle results say models attend best.
Prefer retrieval to stuffing. The trade-off between long context and search is covered in RAG in 2026; for coding agents, token-saving tools ranks the options by evidence.

Measuring it in your own agent

Context rot is a property of your workload, so measure it there. Three cheap signals:

Log input tokens per turn in your traces and chart task success against the context size at which each task ended. If success falls in the upper buckets, your trigger is too high.
Plant a constraint early and test it late. An eval task that states a rule in turn one ("never touch the payments module") and needs it after a compaction tells you whether your summary keeps what matters.
Compare a fresh session. Re-run failed late-stage steps in a new session seeded only with the notes and progress file. If the fresh session succeeds, the problem was the context, not the model.

Those checks fit directly into the trajectory evals and traces you already have; agent observability with OpenTelemetry shows how to capture token counts per span.

Checklist

Treat the context window as a budget for accuracy, not a storage limit.
Paginate and truncate tool output; return handles and summaries.
Clear old tool results first, with exclusions for results that are expensive to recreate.
Compact with domain-specific instructions, keep recent turns verbatim, and append the full response so the compaction block survives.
Keep plans, progress and decisions on disk or in a memory store the agent reads at the start of every session.
Use sub-agents for read-only exploration that returns short summaries; keep writes in one agent with full context.
Put the current goal and constraints at the end of the context.
Measure success against context size and test that early constraints survive compaction.

If your agents degrade on long tasks and you want a second opinion on the harness, see AI engineering.

Context rot in long-running agents: clearing, compaction, memory and sub-agents

Key takeaways