> How to evaluate AI agents: outcome vs trajectory graders, trajectory match modes, pass^k, isolated environments and LLM judges that hold up.
>
> Web page: https://balazscsorba.com/blog/evaluating-ai-agents-trajectories · Language: English · Also available in: [Deutsch](https://balazscsorba.com/de/blog/evaluating-ai-agents-trajectories.md) · [Magyar](https://balazscsorba.com/hu/blog/evaluating-ai-agents-trajectories.md)
> Author: Balázs Csorba · Published: 2026-10-16 · Keywords: evaluating AI agents, agent trajectory evaluation, agent evals, tool call accuracy, trajectory match evaluator, pass^k metric, tau-bench, LLM as a judge for agents, agent regression testing, outcome vs trajectory evaluation, agentevals, tool correctness metric

[Blog](https://balazscsorba.com/blog)/LLMOps & evals

# Evaluating AI agents: grading trajectories, outcomes and reliability

How to evaluate AI agents: outcome vs trajectory graders, trajectory match modes, pass^k, isolated environments and LLM judges that hold up.

[Balázs Csorba](https://balazscsorba.com/about)·October 16, 2026·9 min read

-   Agent evals
-   Trajectories
-   Tool calling
-   pass^k
-   LLM-as-judge

![An agent trajectory of tool calls graded twice: a trajectory grader checks order and arguments, an outcome grader compares the database with the goal state.](https://balazscsorba.com/images/blog/evaluating-ai-agents-trajectories/cover.webp?v=d4f9ed74c3)

## Key takeaways

-   Grade the outcome against the environment state; the agent's final message is the weakest signal of success.
-   Use trajectory checks for invariants that must hold on every path, and avoid demanding one exact path, since agents find valid alternatives.
-   agentevals offers strict, unordered, subset and superset trajectory matching plus per-tool argument rules; DeepEval and Ragas cover similar ground.
-   pass^k, the chance that all k trials succeed, exposes unreliability that pass@k hides; at 75% per trial, three out of three is about 42%.
-   Start each trial in a clean environment, calibrate LLM judges against humans, and build tasks from 20 to 50 real failures.

On this page

1.  [Outcome and trajectory: the two things you grade](https://balazscsorba.com/#outcome-vs-trajectory)
2.  [Code, model and human graders](https://balazscsorba.com/#three-kinds-of-graders)
3.  [Trajectory matching: strict, unordered, subset, superset](https://balazscsorba.com/#trajectory-matching)
4.  [Grade outcomes first, paths only for invariants](https://balazscsorba.com/#grade-outcomes-not-paths)
5.  [pass@k and pass^k: one trial is not a measurement](https://balazscsorba.com/#pass-at-k-vs-pass-hat-k)
6.  [Isolated environments, or the eval lies](https://balazscsorba.com/#isolated-environments)
7.  [LLM judges for trajectories](https://balazscsorba.com/#llm-judges-for-trajectories)
8.  [Capability suites, regression suites and where tasks come from](https://balazscsorba.com/#capability-and-regression-suites)
9.  [Tooling: where the traces and graders live](https://balazscsorba.com/#tooling)
10.  [Checklist](https://balazscsorba.com/#checklist)
11.  [Sources](https://balazscsorba.com/#sources)

Listen to this article

0:000:00

An agent that says "your refund has been processed" and an agent that actually processed the refund look identical in a chat log. Evaluating agents means looking past the final message: at the **outcome** (what changed in the world) and at the **trajectory** (which tools were called, in which order, with which arguments). Grading only the final text is how agents pass evals and fail customers.

This article is the agent-specific follow-up to [LLM evals for product features](https://balazscsorba.com/blog/llm-evals-for-product-features). It covers the vocabulary, the three kinds of graders, trajectory matching with agentevals, DeepEval and Ragas, why pass^k matters more than pass@1 for customer-facing agents, and how to keep the eval environment from lying to you.

## Outcome and trajectory: the two things you grade

Anthropic's January 2026 guide gives a vocabulary worth adopting. A **task** is one test case with inputs and success criteria; each attempt is a **trial**. A **grader** scores some aspect of a trial. The **transcript** (also called trace or trajectory) is the complete record: outputs, tool calls, reasoning, intermediate results. The **outcome** is the final state of the environment. Their example: a flight-booking agent may say "Your flight has been booked", but the outcome is whether a reservation exists in the database.

τ-bench, one of the reference agent benchmarks, grades exactly this way: it compares the database state at the end of a conversation with an annotated goal state. That is the strongest single check you can build, because it does not care what the agent said, only what it did.

Grade the trajectory for rules that must hold on the way, and the outcome for what actually changed. The final message is the least reliable of the three.

## Code, model and human graders

Each grader looks at either the transcript or the outcome. Anthropic's guide splits them into three families, and a good agent eval mixes all three:

Grader

Typical checks

Strengths

Weaknesses

tagchildren

Outcome verification, tool calls used and their parameters, turns taken, token usage, string and regex matches, tests

Fast, cheap, objective, reproducible

Brittle to valid variations; little nuance

tagchildren

Rubric scoring, natural-language assertions, pairwise comparison, reference-based grading

Flexible, handles open-ended output

Non-deterministic, costs more, must be calibrated against humans

tagchildren

Expert review, spot-check sampling, A/B tests

Gold standard, matches expert judgment

Slow and expensive; use it to calibrate the others

Scores can be combined as a weighted sum against a threshold, as a binary all-must-pass, or as a hybrid. My default for business agents: binary on safety-relevant code checks (no refund without verification, ever), weighted on everything else.

## Trajectory matching: strict, unordered, subset, superset

When you have a reference trajectory, the question is how exactly the agent has to follow it. LangChain's open-source [agentevals](https://github.com/langchain-ai/agentevals) package names the four useful answers as `trajectory_match_mode` values:

Mode

Passes when

Use it for

tagchildren

Same messages, same order, same tool calls (message text may differ)

Workflows where order is the rule: verify, then refund

tagchildren

Same tool calls in any order

Information gathering where every source must be consulted

tagchildren

The agent called no tools beyond the reference

Guarding against extra, expensive or risky calls

tagchildren

The agent called at least the reference tools

Minimum required steps, extra lookups allowed

Arguments are matched separately. By default agentevals requires identical arguments; `tool_args_match_mode` can be set to `ignore`, `subset` or `superset`, and `tool_args_match_overrides` takes per-tool rules, including a custom comparator (case-insensitive city names, for example):

```
from agentevals.trajectory.match import create_trajectory_match_evaluator

# Refund flow: the agent may look things up freely, but must make these calls.
evaluator = create_trajectory_match_evaluator(
    trajectory_match_mode="superset",
    tool_args_match_overrides={
        "issue_refund": ["order_id"],          # only order_id must match exactly
        "lookup_order": "ignore",              # any lookup arguments are fine
    },
)
result = evaluator(outputs=agent_messages, reference_outputs=reference_messages)
# {'key': 'trajectory_superset_match', 'score': True, 'comment': None}
```

Other frameworks cover the same ground with different names. DeepEval's `ToolCorrectnessMetric` compares expected and called tools, by name by default, optionally also by input parameters and output, with flags for ordering and exact match. Ragas offers `ToolCallAccuracy` (strict order by default, with a flexible mode), `ToolCallF1` for precision and recall over calls, and `AgentGoalAccuracy` for when only the outcome matters. Pick one framework per project; the concepts transfer. My reviews of [DeepEval](https://balazscsorba.com/tools/deepeval) and [Ragas](https://balazscsorba.com/tools/ragas) cover the trade-offs.

## Grade outcomes first, paths only for invariants

The temptation with trajectories is to demand the exact path you would have taken. Anthropic warns against it: agents regularly find valid approaches the eval designer did not anticipate, so "it's often better to grade what the agent produced, not the path it took." Their own example is striking. Opus 4.5 solved a τ²-bench flight-booking problem by finding a loophole in the policy; it "failed" the eval as written while producing a better result for the user.

So I use trajectory checks for **invariants**, the few rules that must hold on every path, and outcome checks for success. Invariants are usually short and easy to express in code:

```
def check_invariants(calls: list[dict]) -> list[str]:
    """calls: tool calls from the transcript, in order: {"name": ..., "args": {...}}"""
    errors, verified = [], False
    for c in calls:
        if c["name"] == "verify_customer":
            verified = True
        if c["name"] == "issue_refund":
            if not verified:
                errors.append("refund before customer verification")
            if c["args"]["amount"] > 500:
                errors.append("refund above approval limit")
        if c["name"] in {"delete_account", "send_bulk_email"}:
            errors.append(f"forbidden tool: {c['name']}")
    if len(calls) > 25:
        errors.append(f"{len(calls)} tool calls, budget is 25")
    return errors
```

This is binary and cheap, it runs on every trial, and it does not punish creative paths. Combine it with the outcome check against the database, and you have caught the two failures that matter most: the agent did the wrong thing, or it did the right thing in a forbidden way. For deciding which actions need a human gate in the first place, see [human in the loop for AI agents](https://balazscsorba.com/blog/human-in-the-loop-ai-agents).

## pass@k and pass^k: one trial is not a measurement

Agents are non-deterministic, so a task that passes once may fail on the next run. Two metrics describe this. **pass@k** is the probability that at least one of k trials succeeds; it rises with k. **pass^k** is the probability that all k trials succeed; it falls with k. The arithmetic from Anthropic's guide: an agent with a 75% per-trial success rate passes all three of three trials with probability 0.75³ ≈ 42%.

The same agent looks almost perfect under pass@k and unreliable under pass^k. Customer-facing agents should be judged on the second.

τ-bench introduced pass^k precisely because of this gap. In the 2024 paper, even state-of-the-art function-calling agents such as gpt-4o succeeded on fewer than half of the tasks, and pass^8 in the retail domain was below 25%. A coding agent can live with pass@k, because you only need one working patch. A support agent cannot: every customer gets one trial. Run each task at least three to five times and report both numbers.

## Isolated environments, or the eval lies

Outcome grading needs a real environment: a seeded database, a sandbox for file and shell tools, mocked external APIs. Anthropic stresses that each trial must start from a clean state, because shared state causes correlated failures and can inflate scores. Their example: in internal evals Claude gained an unfair advantage on some tasks by reading the git history left by previous trials.

-   **Seed per trial.** Spin up the database from a fixture, run, diff against the goal state, throw it away.
-   **Freeze time and randomness** where tools depend on them ("orders from last week").
-   **Mock the outside world deterministically,** but keep the error responses real: timeouts, 429s and validation errors are part of what you are testing.
-   **Keep the harness identical to production:** same system prompt, same tool descriptions, same limits. Anthropic's point is that you evaluate the model and the harness together.

## LLM judges for trajectories

Some qualities only a model can grade at scale: did the agent ask a sensible clarifying question, did it explain the refusal politely, was the plan reasonable. agentevals ships `create_trajectory_llm_as_judge` with a default trajectory-accuracy prompt; most eval frameworks have an equivalent. The rules for using one are the same as for any judge, only stricter because transcripts are long:

-   **One rubric item per dimension,** each with a concrete pass and fail description.
-   **Give the judge a way out.** Anthropic recommends an explicit "Unknown" answer when the transcript lacks the information, to stop the judge from guessing.
-   **Use partial credit for multi-step tasks.** A support agent that identified the problem and verified the customer but failed the refund is better than one that failed immediately, and your scores should show that.
-   **Calibrate against humans** on a sample before trusting the judge, and re-check when you change the judge model.

## Capability suites, regression suites and where tasks come from

Anthropic separates two kinds of suites. **Capability evals** ask what the agent can do and should start with a low pass rate, giving you a hill to climb. **Regression evals** ask whether it still does what it used to and should sit near 100%. As capability tasks become reliable, they graduate into the regression suite. When a capability suite approaches 100% it stops telling you anything; their example is SWE-bench Verified, where scores started around 30% and frontier models now exceed 80%.

Where tasks come from matters more than how many you have. Anthropic's advice is to start early with 20 to 50 simple tasks drawn from real failures, because early changes have large effects and small samples detect them. My sources, in order of value: production traces where something went wrong (from your [observability pipeline](https://balazscsorba.com/blog/agent-observability-opentelemetry)), the manual checks you already run before a release, and support tickets. Synthetic tasks fill gaps; they should not be the base.

**Read the transcripts**

No metric replaces reading failed trials. Anthropic calls it a critical skill: a failing transcript tells you whether the agent made a real mistake or your grader rejected a valid solution. Budget an hour per week for it, and fix graders as often as you fix prompts.

## Tooling: where the traces and graders live

You need three things: a way to capture full traces, a runner that executes tasks against an isolated environment several times, and a place to store grades over time. OpenAI's platform offers trace grading, scoring agent traces in its dashboard and turning them into eval runs. Open-source frameworks (agentevals with pytest or Vitest integration, DeepEval, Ragas, [Promptfoo](https://balazscsorba.com/tools/promptfoo)) run in CI. Observability platforms such as [Langfuse](https://balazscsorba.com/tools/langfuse) connect production traces to datasets. The choice matters less than the habit: every agent change runs the regression suite before it ships, and the [agent loop's](https://balazscsorba.com/blog/agent-loop-explained) budgets are part of what you assert.

## Checklist

1.  Grade the outcome against the environment state, not the final message.
2.  Express the few must-hold rules as code invariants over the trajectory.
3.  Use reference trajectories with the loosest match mode that still catches the failure (superset or subset before strict).
4.  Run every task three to five times; report pass@1 and pass^k.
5.  Start each trial from a clean, seeded environment that mirrors production.
6.  Use LLM judges with one rubric item per dimension, an Unknown option and partial credit, calibrated against humans.
7.  Keep a capability suite and a regression suite; graduate tasks between them.
8.  Source tasks from real failures, and read failed transcripts every week.

For the retrieval side of agents that search documents, [evaluating RAG](https://balazscsorba.com/blog/rag-evaluation-metrics) covers the metrics. If you want help setting up agent evals for your team, see [AI engineering](https://balazscsorba.com/expertise/ai-engineer).

## Sources

1.  [Anthropic: Demystifying evals for AI agents (Jan 2026)](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents)
2.  [Yao et al.: τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains (2024)](https://arxiv.org/abs/2406.12045)
3.  [Barres et al.: τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment (2025)](https://arxiv.org/abs/2506.07982)
4.  [LangChain: agentevals (GitHub)](https://github.com/langchain-ai/agentevals)
5.  [DeepEval docs: Tool Correctness](https://deepeval.com/docs/metrics-tool-correctness)
6.  [Ragas docs: Agentic or tool use metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/agents/)
7.  [OpenAI API docs: Trace grading](https://platform.openai.com/docs/guides/trace-grading)

## Frequently asked questions

What is trajectory evaluation for AI agents?

Trajectory evaluation grades the sequence of steps an agent took: which tools it called, in which order and with which arguments. It complements outcome evaluation, which checks the final state of the environment, such as whether a refund record exists. Use trajectories mainly for rules that must hold on every path.

What is the difference between pass@k and pass^k?

pass@k is the probability that at least one of k attempts succeeds, so it rises as k grows. pass^k is the probability that all k attempts succeed, so it falls. Coding agents can use pass@k because one working patch is enough; customer-facing agents should be judged on pass^k because every customer gets one attempt.

Should an agent eval require an exact sequence of tool calls?

Only where the order is the rule, such as verifying a customer before a refund. Otherwise use looser modes like superset or subset, or grade the outcome. Anthropic reports a case where a model failed an eval as written by finding a policy loophole that produced a better result for the user.

How many tasks do I need to start evaluating an agent?

Anthropic suggests 20 to 50 simple tasks drawn from real failures as a good start, because early changes have large effects that small samples detect. Run each task several times and grow the suite as the agent matures.

Can I use an LLM as a judge for agent transcripts?

Yes, for qualities code cannot check, such as whether a clarifying question was sensible. Use one rubric item per dimension, give the judge an Unknown option, award partial credit on multi-step tasks and calibrate it against human grades on a sample.

Written by Balázs Csorba

Senior fullstack & AI engineer in Styria, Austria – 10+ years of Vue, Nuxt, Node.js and PHP, now building tooling for AI agents.

[AI engineering & MCP servers →](https://balazscsorba.com/expertise/ai-engineer)[About me →](https://balazscsorba.com/about)

## More articles

-   [Structured outputs with JSON Schema: what the guarantee covers, and what it does not](https://balazscsorba.com/blog/structured-outputs-json-schema-llm)
-   [LLM model migrations without regressions: deprecations, evals, shadow traffic](https://balazscsorba.com/blog/llm-model-migration-regression-testing)
-   [Semantic caching for LLM apps: when reusing answers is safe, and when it is not](https://balazscsorba.com/blog/semantic-caching-llm-responses)
-   [Self-hosting LLMs for GDPR: when it is required and what it costs](https://balazscsorba.com/blog/self-hosted-llm-gdpr-cost)

## Sounds like what you need?

Tell me about your project or role – I’d love to hear from you.

[Get in touch](mailto:contact@balazscsorba.com) [Connect on LinkedIn](https://www.linkedin.com/in/balazs-csorba)