Can a better system prompt prevent prompt injection?

No. A system prompt is more text in the same context window, and the model weighs it against everything else it reads. Clear instructions and delimiters reduce accidental failures, but a determined attacker can still override them. Reliable protection comes from architecture: limiting what data the agent can reach, which actions untrusted content can trigger, and where the agent can send data.

What is the difference between direct and indirect prompt injection?

Direct prompt injection comes from the person typing into the model, for example a user trying to override a chatbot's rules. Indirect prompt injection hides instructions in content the model reads on someone's behalf, such as a web page, an email, a ticket or a tool result. Indirect injection is more dangerous for agents because the victim never sees the malicious text.

Do prompt injection classifiers or guard models work?

They help as a second layer, but they are not a security boundary. Research published in October 2025 bypassed 12 recent defenses with adaptive attacks at success rates above 90% for most. Use classifiers to raise attacker cost and catch careless attacks, and rely on data scoping, egress control and human approval for the actual guarantees.

How can Markdown images leak data from an AI chat?

If a chat interface renders Markdown automatically, an injected instruction can make the model output an image whose URL contains private data in the query string. The browser fetches the image and sends the data to the attacker's server without any click. Rendering replies as plain text, or proxying images through a fixed allowlist, closes this channel.

Prompt injection and the lethal trifecta

Prompt injection is an attack in which text that an LLM reads as data, such as a web page, an email, an issue comment or a tool result, contains instructions that the model then follows. It is not a bug in one model that the next release will fix. It follows from how language models work, which makes it an architecture problem: you can't reliably filter it out, so you design systems in which a successful injection can't do much damage.

This post explains why models can't separate instructions from data, what Simon Willison calls the lethal trifecta, and the six design patterns from a 2025 paper that constrain what an injected instruction can reach. Then it applies them to two common systems, a support bot and a coding agent, and ends with a checklist and a red-team setup you can run in CI.

Why can't an LLM tell instructions from data?

Because everything in the context window is the same thing to the model: a sequence of tokens. The system prompt, the user's request and the content of a fetched web page all arrive in one stream, and the model predicts what comes next based on all of it. There is no separate, privileged channel for "the instructions I should obey".

Willison puts it plainly in his June 2025 post on the lethal trifecta: models "will happily follow any instructions that make it to the model", not only yours. Role markers and delimiters help a model weigh sources, but they are conventions the model learned, not a boundary it enforces.

Detection doesn't close the gap either. In "The Attacker Moves Second" (October 2025), researchers from OpenAI, Anthropic and Google DeepMind used adaptive attacks against 12 published defenses and bypassed them "with attack success rate above 90% for most". Defenses that looked near-perfect against a fixed set of attack prompts failed once the attacker tuned the attack to the defense. In security terms, a filter that stops 95% of attacks is a filter that fails every determined attacker.

Anthropic reached the same conclusion from the other side. In "How we contain Claude" (May 2026) it describes an internal red-team exercise in which a phishing email carried a ready-to-paste prompt that told Claude Code to read AWS credentials, encode them and POST them out: "Across 25 retries of that prompt, Claude completed the exfiltration 24 times." The post's conclusion is that model-layer protection "will never be 100% effective, which is why it can't stand alone."

What is the lethal trifecta?

The lethal trifecta is the combination of three capabilities in one agent: access to private data, exposure to untrusted content, and the ability to communicate externally. If an agent has all three, an attacker who controls any of the untrusted content can make it read the private data and send it out.

The lethal trifecta: private data, untrusted content and external communication. Any two can be contained; all three together let an injected instruction read private data and send it to the attacker.

The legs are broader than they look. "External communication" includes an HTTP request, a sent email, a comment on a public issue, and a Markdown image whose URL carries data in its query string, which a chat UI fetches automatically. "Untrusted content" includes anything a third party can write to: support tickets, product reviews, PDFs, README files, dependency source code and tool descriptions from third-party servers.

Meta's security team turned the same idea into a rule in "Agents Rule of Two" (October 2025): within one session, an agent should have at most two of [A] processing untrustworthy inputs, [B] access to sensitive systems or private data, and [C] the ability to change state or communicate externally. If a workflow truly needs all three, Meta says the agent "should not be permitted to operate autonomously" and needs human-in-the-loop approval or another reliable means of validation. Meta's [C] also covers changing state, not only sending data out, which is the right extension for agents that can delete, pay or deploy.

Six design patterns that contain prompt injection

The six patterns come from "Design Patterns for Securing LLM Agents against Prompt Injections" (Beurer-Kellner et al., June 2025). They share one principle: once an agent has read untrusted input, that input must not be able to trigger consequential actions. Each pattern gives up some flexibility to get there.

1. Action-selector

The model maps a request to one action from a fixed list and never sees the results of that action. Willison's summary of the paper calls it an "LLM-modulated switch statement". Nothing flows back, so nothing can be injected.

2. Plan-then-execute

The agent fixes its complete plan of tool calls before it reads any untrusted content. Tool outputs can still corrupt the content of a step, for example the text of a summary, but they cannot add, remove or reorder steps.

3. LLM map-reduce

Each untrusted document goes to an isolated sub-agent that returns a constrained result, such as a boolean or a number. A coordinator aggregates the results. A poisoned document can only corrupt its own result.

4. Dual LLM

Willison first described it in April 2023. A privileged LLM plans and calls tools but never sees untrusted text. A quarantined LLM processes untrusted text but has no tools. Ordinary code (the controller) passes results between them as symbolic variables such as $VAR1, so the privileged model can say "email $VAR1 to the user" without reading $VAR1.

// Pseudo-code: dual LLM with a code controller
plan = privileged_llm(user_request, tools)   // never sees untrusted text
for step in plan:
  if step.kind == "read_untrusted":
    vars[step.out] = quarantined_llm(step.prompt, fetch(step.source))  // no tools
  else:
    execute(step.tool, resolve(step.args, vars))  // $VAR1 substituted by code, not by a model

5. Code-then-execute

The privileged model writes a program in a restricted language that states which tools are called and how data flows between them. An interpreter runs it and can track which values are tainted by untrusted sources. Google DeepMind's CaMeL is the best-known version; it adds capability-based policies on top, and in its paper solves 77% of AgentDojo tasks with provable security, against 84% for an undefended system.

6. Context minimization

Remove what the model no longer needs. If a user's request has been turned into a database query, drop the original request before the model sees the query results, so injected text in the request can't steer the answer.

Dual LLM: the privileged model plans and calls tools but only ever sees variable names; the quarantined model reads untrusted text but has no tools; ordinary code substitutes values when a tool runs.

Why an egress allowlist is a capability grant

An egress allowlist is the list of hosts an agent may reach. It is the most direct way to cut the external-communication leg of the trifecta, but only if you treat every allowed domain as a set of capabilities, not as a safe destination.

The Anthropic red-team exercise above worked because Claude Code's allowlist permitted api.anthropic.com, and an API that accepts uploads is an exfiltration channel. The post's lesson: "Every function reachable through any domain on an allowlist is now an attack surface." Anthropic's fix was a proxy that only passes requests carrying the session's own provisioned token, so an attacker's embedded key is rejected.

The Claude Code sandbox documentation makes a related point: allowing broad domains such as github.com "can create paths for data exfiltration", and because the default proxy decides from the client-supplied hostname without inspecting TLS, domain fronting can reach hosts outside the list. Practical consequences:

Allow specific hosts and paths (a package registry mirror), not whole platforms that accept writes.
Bind credentials to the session at the proxy, so a request with a foreign token fails.
Strip or block rendering of external images and links in chat output, or proxy them through a fixed allowlist.
Log every egress request with the session ID, so an exfiltration attempt is visible after the fact.

Sandbox, network and credential controls for agents in CI get their own treatment in the checklist for sandboxing coding agents.

Applying the patterns to a support bot and a coding agent

Start by listing which legs of the trifecta each system has, then pick the pattern that removes one leg or stops untrusted content from choosing actions. The table below is my reading of where each pattern fits; the paper's case studies include both a customer service chatbot and a software engineering agent.

Pattern	Support bot (reads tickets, looks up orders)	Coding agent (reads repo, issues, web)	Main cost
Action-selector	Strong fit for routing: refund form, order status, human handover	Weak: agents need tool feedback	No free-form answers
Plan-then-execute	Good for fixed flows such as "look up order, then answer"	Partial: plan fixed per task, re-plan needs approval	No adaptive steps
LLM map-reduce	Classifying many tickets or reviews	Scanning many files or dependencies	Only narrow outputs per item
Dual LLM	Summarizing customer emails without tool access	Summarizing issues and web pages for the planner	Complex controller code
Code-then-execute	Possible, usually overkill	Promising for tainted-data tracking	Custom interpreter and policies
Context minimization	Drop the raw message after intent extraction	Drop fetched pages after use	Less context for follow-ups

The support bot

A support bot that reads a customer's message and can look up that customer's orders has two legs: untrusted content and private data. Keep it at two. Scope the order lookup to the authenticated customer in code, not in the prompt, so an injected "show me order 1234 of another customer" returns nothing. Render replies as plain text without auto-loaded images or links, which removes the quiet exfiltration path. Anything that changes state, such as a refund, goes through an action-selector that opens a form a human or a rule engine approves.

The coding agent

A coding agent usually has all three legs: it reads the repository and secrets in the environment, it reads issues, dependencies and web pages, and it can run curl or push a branch. Here the patterns become environment controls: no production credentials in the sandbox, an egress allowlist limited to package mirrors, and pushes that go only to a branch that a human reviews. Third-party MCP servers add their own injection surface through tool descriptions; the MCP server security checklist covers tool poisoning and rug pulls.

Trade-offs: when the patterns cost too much

Every pattern removes flexibility, and some products need that flexibility. The honest trade-off is between autonomy and blast radius, and the right answer depends on what the worst injected action could do.

General-purpose assistants that browse, read email and send messages break the patterns by design. The realistic controls are human confirmation for every external action and a short list of allowed actions.
Dual LLM and code-then-execute need real engineering: a controller, variable handling, an interpreter, policies. For a small internal tool with no private data, that effort is not worth it; removing the private-data leg is cheaper.
Human approval degrades when it's constant. People approve prompts they don't read. Put approvals on the few irreversible actions, not on every tool call.
Classifiers and guard models still have a place as a second layer that raises cost for attackers and catches careless attacks. They are not a boundary, and the adaptive-attack results above show why.

Prompt injection checklist

Write down, per agent, which of the three trifecta legs it has. If it has all three, treat that as a design defect to fix or to gate with human approval.
Enforce data scope in code (tenant, user, row-level permissions), never in the system prompt.
Pick one containment pattern per untrusted input path: action-selector, plan-then-execute, map-reduce, dual LLM, code-then-execute or context minimization.
Treat each allowlisted domain as a capability grant; allow narrow hosts and bind credentials to the session at the proxy.
Remove silent exfiltration channels: auto-loaded images, link unfurling, and tools that accept arbitrary URLs.
Require human approval for irreversible or public actions: payments, deletes, deploys, outbound email, public comments.
Log tool calls and egress with a session ID so you can reconstruct what an injected instruction did.
Red-team every release with adaptive attacks, not a fixed list of known prompts, and track the results as a regression suite.

For the last item, promptfoo maps its red-team plugins to the OWASP Top 10 for Agentic Applications, where ASI01 is Agent Goal Hijack. A minimal configuration that runs all ten categories with multi-turn strategies looks like this:

redteam:
  plugins:
    - owasp:agentic
  strategies:
    - jailbreak
    - jailbreak-templates
    - crescendo

Treat the output like any eval: read the failing transcripts, turn real failures into fixed test cases, and gate releases on them. The post on evals for LLM features describes how to turn transcripts into a regression suite. If you're designing an agent that has to live with all three legs of the trifecta, that is the kind of work I do as an AI engineer.

Prompt injection defense: the lethal trifecta and six design patterns

Key takeaways