Why a longer system message will not save you
The model sees one context. Your instructions, the retrieved document, the tool output and the user’s message arrive as the same kind of thing: text. There is no privileged channel that says these words are policy and those words are data, however firmly you phrase it. Every mitigation built purely on wording is a probabilistic filter over an adversarial input, and adversarial inputs are cheap to generate.
This is not a reason for despair, but it does relocate the problem. We stopped treating injection as a prompt-hardening exercise about two years ago and started treating it the way we treat SQL injection: an input-trust boundary question, solved by controlling what the untrusted input is able to reach.
The model cannot tell your policy from a paragraph in a PDF. Both are just text in the same window, competing for attention.
The severity is in the tools, not the text
An injected instruction on its own is a curiosity. An injected instruction in a system that can send email, issue a refund, update a record or call an internal API is an incident. The distinction is nothing to do with the model and everything to do with what you connected.
So the first exercise on any agent review is an inventory of capability, sorted by what it would cost to be wrong. Read-only lookups sit at the bottom; anything that moves money, changes entitlements or communicates externally sits at the top. That list, not the prompt, is the attack surface — and shortening it is usually the highest-value change available.
Where the controls actually go
These are the controls we install by default. None of them are exotic; all of them are structural rather than textual, which is why they hold when the wording of an attack changes.
Your own document store is an attack path
The scenario people underrate: an attacker does not need access to your agent, only to something your agent will read. A supplier PDF, a support ticket, a calendar invitation, a page on an internal wiki that anyone can edit. Instructions placed there wait patiently until a retrieval step pulls them into context.
We found a live example of this in a procurement pilot — not a real attack, a curious employee — where text hidden in a submitted document tried to have the assistant recommend that supplier. It did not work, because the recommendation step had no authority to act. That is the whole argument for structural defence in one anecdote.
An attacker does not need your agent. They need something your agent will read, and a supplier PDF is easier to reach than your network.
Test it like a control, because it is one
Injection resistance belongs in the evaluation suite next to accuracy, with cases drawn from your own corpus and your own tools. Generic jailbreak lists are a starting point and nothing more; the interesting attacks are specific to what your system can do.
We also insist on a documented residual position. Some injection risk remains after every mitigation, and the honest artefact says so: here is what an attacker could still achieve, here is why we accept it, here is what would change our mind. Auditors respond well to that document. They respond badly to a claim that the system is immune.
The design question
Stop asking how to make the model resist instructions in its context. Ask what the worst instruction in that context could actually cause the system to do — then remove the capability that makes the answer frightening.
Talk to our security team