What is the difference between a guide and a sensor in a coding agent harness?

A guide is feedforward: it steers the agent before it acts, for example an AGENTS.md file, a skill, type definitions or a scaffolding script. A sensor is feedback: it checks the result after the agent acts, for example tests, a type checker, a linter or an AI review, and returns the findings so the agent can correct itself.

Can I trust tests written by a coding agent?

Not by coverage alone. Agent-written tests can reach high coverage with weak assertions. Check them by making the agent show a failing run before the passing one, by reverting the fix and confirming the regression test fails, and by running mutation testing on the changed lines to see whether the tests catch small deliberate bugs.

How do I make a coding agent always run the tests before committing?

Don't rely on an instruction alone, because agents sometimes skip steps written in rules files. Enforce the gate with a pre-commit hook, an agent hook that blocks the commit command until the checks pass, or a required CI check. Keep the instruction too, so the agent runs the checks early and fixes failures itself.

Is mutation testing too slow for CI?

On a whole codebase it often is, but you don't need that on every pull request. Tools such as Infection for PHP can mutate only the lines a branch changed, compared with the base branch, which keeps the run short. Run full mutation analysis on a schedule instead.

Harness engineering for coding agents

Harness engineering is the work of building everything around a coding agent's model (instructions, tools, tests, linters, reviews) so that what the agent produces is correct and maintainable before a human looks at it. The model writes the code. The harness decides whether that code is any good, and tells the agent when it isn't.

This article uses Birgitta Böckeler's framework of guides and sensors, then gets practical: which checks to run where, how red/green TDD works with an agent, how mutation testing checks the tests the agent wrote, and why behaviour is still the weak spot. It ends with a checklist for making agent pull requests mergeable.

What is harness engineering?

Harness engineering treats the environment around the model as the thing you design. Böckeler's article "Harness engineering for coding agent users" (April 2026) sums it up as "Agent = Model + Harness".

She separates two harnesses. The agent builder's harness is what ships inside Claude Code, Codex or opencode: the system prompt, the tool loop, the sandbox. The user's harness is what you add around it for your codebase: rules files, skills, test commands, linters, review steps. You can't change the first much. The second is yours, and it's where most of the quality difference between teams comes from.

Some codebases take a harness better than others. Böckeler calls this harnessability: a "strongly typed language naturally has type-checking as a sensor", and clear module boundaries make architecture rules possible. A codebase with no tests and no types gives an agent nothing to check itself against, so it has to guess.

Guides and sensors: feedforward and feedback

Guides steer the agent before it acts; sensors check the result after it acts, so the agent can correct itself. Both come in two kinds: computational (run by the CPU, deterministic) and inferential (run by a model, semantic but probabilistic).

In Böckeler's words, guides "anticipate the agent's behaviour and aim to steer it before it acts", while sensors "observe after the agent acts and help it self-correct". Computational controls are "deterministic and fast, run by the CPU": tests, linters, type checkers. Inferential controls are semantic analysis and AI code review: slower, more expensive and non-deterministic, but able to judge things a linter can't.

The harness as a two by two grid: guides steer before the agent acts, sensors check afterwards, and each can be computational (deterministic) or inferential (model-based).

Property	Computational control	Inferential control
Examples	Tests, type checker, linter, dependency rules, mutation testing	AI code review, review sub-agent, rules in AGENTS.md or skills
Result	Same input, same answer	Can differ between runs
Speed and cost	Fast, cheap to repeat	Slower, costs tokens per run
Catches	Structural problems: types, style, coverage, forbidden imports	Semantic problems: naming, design, missing edge cases
Can block a merge on its own?	Yes	Better as advice to the agent or a human

Böckeler groups harnesses by what they protect. Maintainability is the most developed: computational sensors reliably catch "duplicate code, cyclomatic complexity, missing test coverage, architectural drift". Architecture fitness covers performance requirements and conventions, checked with fitness functions. Behaviour, whether the software does what it should, is the gap, and it gets its own section below.

Where should sensors run?

Run the fast sensors inside the agent's loop after every change, the full gate before every commit, and the slow ones in CI. The earlier a sensor fires, the cheaper the correction: the agent fixes it in the same session, before a reviewer ever sees it.

Böckeler calls this keeping quality left: fast linters pre-commit, broader review after commit, expensive analyses such as mutation testing after integration. In her follow-up, "Maintainability sensors for coding agents" (May 2026), she tried ESLint, dependency-cruiser, Semgrep, coverage, Stryker and GitLeaks as sensors, and found that computational tools work best at file level while model-based review is better at cross-module concerns. She also reports the practical problem: "I had to ask the agents many, many times why it had not run the sensors check."

That line is the argument for enforcement. A rule in a Markdown file is a guide; the agent may or may not follow it. Claude Code's memory docs say the same about their own rules files: Claude treats them "as context, not enforced configuration", and to block an action regardless of what the model decides, you use a hook. My own gate is deliberately boring: the full PHPUnit suite and PHPStan run before every commit, and a skipped or partial run is reported as exactly that, never as green.

Sensors close the inner feedback loop so the agent fixes structural problems itself; the human review loop is reserved for intent and design.

Red/green TDD with coding agents

Red/green TDD means the agent writes a test, runs it and watches it fail, then writes the code and watches it pass. The failing run is the point: it proves the test exercises the new behaviour.

Simon Willison lists it in his Agentic Engineering Patterns guide as a short prompt, "Use red/green TDD", and gives the reason: "If you skip that step you risk building a test that passes already, hence failing to exercise and confirm your new implementation." A companion pattern, "First run the tests", makes the agent discover how the suite runs at the start of a session, which makes it far more likely to run it again later.

For bug fixes I use a stricter version. The regression test must pass with the fix, fail when only the fix is reverted, and pass again when it goes back in. The agent does the revert itself and reports both runs. This catches tests that assert on the wrong thing, and it's cheap: two extra test runs. The workflow around it is described in how I use agent skills for bug fixes.

Anthropic's "Effective harnesses for long-running agents" applies the same idea to whole projects: a feature list where every feature starts as "passes": false, one feature per session, and a rule that "it is unacceptable to remove or edit tests". The red state is written down before any code exists.

Mutation testing: checking the tests the agent wrote

Mutation testing makes small changes to your production code and reruns the tests. If the tests still pass, they didn't notice the change, and they're weaker than their coverage number suggests.

Stryker describes it this way: mutants are "automatically inserted into your production code. Your tests are run for each mutant. If your tests fail then the mutant is killed. If your tests passed, the mutant survived." Stryker covers JavaScript and TypeScript, C# and Scala. For PHP, Infection reports a Mutation Score Indicator (MSI) and can fail a CI job below a threshold.

This matters more once agents write most tests. An agent asked to "add tests" will reach high line coverage easily; whether the assertions would catch a real bug is a different question. Böckeler makes the same observation: coverage alone masks weak tests, so mutation testing becomes more important when AI writes them. Mutation runs are slow on a whole codebase, so scope them to the change:

# CI step: mutate only the lines this PR touched (PHP, Infection)
git fetch --depth=1 origin $GITHUB_BASE_REF
infection --git-diff-lines --git-diff-base=origin/$GITHUB_BASE_REF --min-covered-msi=80
# 80 is an example threshold; start from your current score and raise it

Surviving mutants are also good agent input. Hand the list back with "each of these survived; add or tighten assertions so they are killed, without changing production code", and the sensor turns into a feedback loop the agent can close itself.

The weak spot: behaviour harnesses, and when not to over-build

Behaviour is where harnesses are weakest, because the usual behaviour check is a test suite the agent wrote from a spec the agent read. Böckeler's verdict on that setup: it puts "a lot of faith into the AI-generated tests, that's not good enough yet."

Three things help, none of them free:

Human-owned acceptance tests. A small set of end-to-end tests that a person wrote or approved, which the agent may run but not edit.
Tests through the real interface. Anthropic's long-running harness found browser automation critical so the agent verified features "as a human user would". Playwright plays this role for me.
Separate review of the test diff. When code and tests change together, a reviewer reads the tests first.

The trade-off is cost and noise. Every sensor adds time to the loop, and inferential sensors add tokens and false positives. A review sub-agent that flags ten style nits per PR trains everyone, human and agent, to ignore it. Don't build a harness for a throwaway script. Do build one for code that will be maintained, and grow it from real failures: each time an agent PR needs a human correction, ask which guide or sensor would have caught it. Böckeler's caveat also holds: humans bring judgment and accountability as an "implicit harness", and a written harness "can only go so far". How to keep human review from becoming the bottleneck is covered in reviewing AI-generated pull requests.

Harness engineering checklist

Write the guides: build and test commands, conventions and forbidden patterns in AGENTS.md or skills.
Make fast sensors run in the loop: type checker, linter and the affected tests after each change.
Enforce the gate with hooks or CI, not with a sentence in a prompt: full suite and static analysis before every commit.
Require red/green: the agent shows the failing run before the passing one; for bug fixes, the test fails when the fix is reverted.
Add mutation testing on the diff and feed surviving mutants back to the agent.
Protect acceptance tests that a human owns; the agent may run them, not edit them.
Keep inferential review advisory and tune it until its comments are worth reading.
Turn every human correction into a guide or sensor, so the same mistake doesn't reach review twice.

If you're setting up a harness like this for a team, see AI engineering.

Harness engineering: guides and sensors that make agent PRs mergeable

Key takeaways

What is harness engineering?

Guides and sensors: feedforward and feedback

Where should sensors run?

Red/green TDD with coding agents

Mutation testing: checking the tests the agent wrote

The weak spot: behaviour harnesses, and when not to over-build

Harness engineering checklist

Sources

Frequently asked questions

Sounds like what you need?