Reasoning effort and thinking budgets: how much should a model think?

A year ago, controlling how much a model reasoned meant picking a number: a thinking budget of 4,000 or 16,000 tokens. In 2026 the three big APIs have converged on something different. Claude and OpenAI expose an effort level, Gemini a thinking_level, and the model decides per request whether to think at all. That is easier to configure and harder to reason about, because the knob no longer maps to a token count.

This post covers what each API actually offers, what thinking costs on the invoice, why max_tokens is the only hard limit you have, why more thinking can make answers worse, and how I pick a level for a feature: an effort sweep on a real eval set, then a fixed setting per conversation so the prompt cache survives.

What thinking tokens are and what you pay for

Reasoning models generate tokens before, and on current models also between, their visible answer and tool calls. On every major API those tokens are billed as output, the expensive side of the price sheet. OpenAI says plainly that reasoning tokens are not visible through the API but occupy context and are billed as output tokens. Gemini prices a thinking response as the sum of output and thinking tokens. Anthropic bills the full internal thinking even when the response only carries a summary.

On Claude, the display setting changes only what you see. With summarized you get summary text, with omitted the thinking field is empty, and in both cases the billed output tokens are the full thinking the model generated. Anthropic warns that the billed count does not match the visible count. There is a second, quieter cost: Claude Opus 4.5 and models numbered 4.6 and higher keep thinking blocks from earlier turns in context and bill them as input on every following request.

The controls on Claude, OpenAI and Gemini

The three APIs look similar from a distance and differ in the details that break code: where the parameter lives, which values a model accepts and what the default is. The defaults matter most, because an omitted parameter is a decision too.

Not every model accepts every value. On Claude, not every model that supports max supports xhigh, and the effort docs carry per-model recommendations that override the general table: Opus 4.7 and 4.8 recommend starting at xhigh for coding and agents, while Sonnet 5.5 suggests medium for well-specified agentic coding. On OpenAI, GPT-6 Astra rejects none with HTTP 400. On Gemini, gemini-3-pro-preview only knows low and high.

From budget_tokens to effort

The old Claude control, thinking: {type: "enabled", budget_tokens: N}, is deprecated on the 4.6 models and rejected with a 400 error on Claude 4.7 and later. It still matters if you run Claude 4.5 or earlier, where it is the only thinking mode: the budget must be at least 1,024 tokens and below max_tokens, and Anthropic recommends batch processing for budgets above 32k to avoid timeouts. The migration is small in code and large in behaviour.

// before: fixed budget, Claude thinks on every request
{ "model": "claude-sonnet-4-6", "max_tokens": 16000,
  "thinking": { "type": "enabled", "budget_tokens": 10000 } }

// after: adaptive thinking, depth controlled by effort
{ "model": "claude-sonnet-4-6", "max_tokens": 16000,
  "thinking": { "type": "adaptive" },
  "output_config": { "effort": "high" } }

With a fixed budget, Claude thinks on every request. With adaptive thinking it decides per request, and at lower effort it may skip thinking entirely on easy inputs, so do not write code that assumes every assistant turn starts with a thinking block. Effort also reaches beyond thinking: it shapes text and tool calls, and lower levels make fewer and terser tool calls. On Opus 5.5, adaptive thinking cannot be switched off at all; thinking: {type: "disabled"} returns a 400.

max_tokens is the only hard cap

Effort is guidance, not a budget. Anthropic calls it a behavioural signal, and the hard ceiling is max_tokens, which covers thinking and answer together. If a hard request thinks long, it can run into that ceiling before writing the answer. Claude then stops with stop_reason: "max_tokens". OpenAI returns status: "incomplete" with reason max_output_tokens, possibly before any visible output, and you still pay for the reasoning. Gemini behaves the same way and bills the thinking tokens it generated.

OpenAI recommends reserving at least 25,000 tokens for reasoning and output when you start, and Anthropic suggests starting around 64k for xhigh and max on its larger models. When you see truncation, decide which fix applies: if the truncated requests needed the reasoning, raise the cap; if they were overthought, lower the effort. Either way, handle the truncated case in code instead of shipping a half answer.

// Claude: treat truncation as a failure, not as an answer
const res = await client.messages.create({
  model: "claude-opus-5-5",
  max_tokens: 32000,
  output_config: { effort: "high" },
  messages,
});
if (res.stop_reason === "max_tokens") {
  metrics.increment("llm.truncated", { effort: "high" });
  // retry once with a larger cap, or fall back to a lower effort
}
const thinking = res.usage.output_tokens_details?.thinking_tokens ?? 0;

More thinking is not always better

The intuitive model, more reasoning means better answers, holds on hard problems and fails at the edges. Chen and colleagues documented overthinking in o1-style models in late 2024: excessive compute spent on simple problems with minimal benefit, such as long chains for 2+3. In July 2025, Anthropic-affiliated researchers published tasks where longer reasoning lowers accuracy, an inverse scaling between test-time compute and quality, including models distracted by irrelevant information and drifting toward spurious correlations.

The vendor docs say the same in operational terms. Anthropic's guidance for Opus 4.7 notes that max adds significant cost for small gains on most workloads and can lead to overthinking on structured-output or less intelligence-sensitive tasks. OpenAI recommends xhigh only when evals show a clear benefit. For an extraction or classification feature, the right answer is often the lowest level that holds quality, combined with structured outputs so the format does not depend on how long the model thought.

Picking a level: the effort sweep

Every vendor ends its guidance with the same instruction: test on your own workload. Anthropic explicitly says to run a fresh sweep instead of carrying settings over from an earlier model, because levels are recalibrated between generations. In practice that is a small script: the same eval set, the same model, every supported level, and four numbers per level.

for (const effort of ["low", "medium", "high", "xhigh"]) {
  const runs = await Promise.all(evalSet.map(async (c) => {
    const t0 = Date.now();
    const r = await client.messages.create({
      model, max_tokens: 64000, output_config: { effort },
      messages: c.messages,
    });
    return { ms: Date.now() - t0, out: r.usage.output_tokens,
             truncated: r.stop_reason === "max_tokens",
             score: await grade(c, r) };
  }));
  report(effort, runs); // pass rate, mean output tokens, p95 ms, truncation rate
}

Read the result as a curve, not a winner. If medium passes the same cases as high at a fraction of the output tokens, medium wins. If high fixes the cases that matter to the business, the extra tokens are cheap. The grading is the hard part, and it is the same work as any LLM eval for a product feature: labelled cases from real traffic and a grader you trust.

Routing effort per request without breaking the cache

One level per feature is the baseline. The next step is to vary it per request: low for a routine confirmation, high for the planning step. Anthropic lists dynamic effort as a best practice, with a catch. The resolved effort value is rendered into the prompt, so changing the top-level value between requests invalidates cache breakpoints. Anthropic's own example shows a full cache read on the second request and a fresh cache write on the third, after effort dropped from medium to low.

Two ways around it. Across workloads, route at the feature boundary, where conversations do not share a cache anyway; the routing logic is the same as model routing. Inside a conversation, current Claude models such as Opus 5.5, Sonnet 5.5 and Haiku 5.5 support a per-message effort change in beta, behind the mid-conversation-output-config-2026-07-01 header: an empty system message carrying output_config.effort changes the level from the next user turn and keeps the earlier prefix cached.

Prompt-based steering is the third lever. Anthropic documents phrases such as "Answer directly without deliberating." for routine turns, but warns that it is wording-sensitive and less reliable than effort, and on Haiku 5.5 the discouraging phrase did not reliably reduce thinking. I use it only after the level is right.

Agent loops: interleaved thinking and task budgets

In an agent, one user message becomes many requests, and max_tokens only caps each request, not the whole turn. Current models think between tool calls automatically. Anthropic notes that the first request after new user input typically carries most of the reasoning, and follow-ups that only process tool results can skip thinking even at xhigh. OpenAI asks you to pass reasoning items back with function call outputs, through previous_response_id or the full output items, so the model can continue its reasoning without redoing it.

For the whole loop, Claude has task budgets in beta (task-budgets-2026-03-13): output_config.task_budget with type: "tokens" and a total of at least 20,000. The model sees a server-side countdown and paces itself to finish gracefully. It is advisory, not enforced, and a budget that is clearly too small can make Claude scope the task down or decline it. Anthropic suggests sizing it from the p99 of measured per-task spend. Effort tunes depth per step; the task budget tunes total breadth. The same agent-loop mechanics are covered in the agent loop, explained.

Measure thinking like any other cost

All three APIs report thinking separately, so there is no excuse for guessing. Claude returns usage.output_tokens_details.thinking_tokens, which is always less than or equal to output_tokens and appears on the final message_delta event when streaming. OpenAI reports output_tokens_details.reasoning_tokens. Gemini reports total_thought_tokens. Put the field on every trace next to the effort level and model id.

Three charts are enough: thinking tokens per request by feature, truncation rate by effort level, and quality score by effort level from your sampled evals. When thinking per request creeps up after a prompt change, you see it the same day instead of on the invoice. The tracing side is in observability for LLM agents with OpenTelemetry.

Sources

1. Claude Platform Docs: Effort
2. Claude Platform Docs: Extended thinking (manual budget_tokens mode)
3. Claude Platform Docs: Steering thinking (cost, caching, pricing)
4. Claude Platform Docs: Task budgets
5. OpenAI API Docs: Reasoning models
6. Google AI for Developers: Gemini thinking
7. Chen et al.: Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs (arXiv 2412.21187)
8. Gema et al.: Inverse Scaling in Test-Time Compute (arXiv 2507.14417)