Agent securityMiddleLesson 206 min read

Guardrails, and why they leak

Filters, classifiers and safety prompts are worth having. Just be honest about what they are: speed bumps, not walls.

🧪 Lab · 20 min📝 Lesson test
Lesson in motion

In 60 seconds

Guardrails, and why they leak

Filters, classifiers and safety prompts are worth having. Just be honest about what they are: speed bumps, not walls.

1/6
In simple words
A metal detector at the door catches most knives. It does not catch a knife made of plastic. Useful — but you would not remove the locks because you installed one.
A guardrail is anything that inspects input or output and blocks the bad ones. They live in three places:
  1. 1

    Input guardrails

    Scan what arrives before the model sees it. Known attack phrases, hidden characters, suspicious formatting, off-topic requests.
  2. 2

    Output guardrails

    Scan what the model produced before anyone acts on it. Leaked secrets, disallowed content, URLs pointing somewhere strange.
  3. 3

    Action guardrails

    Check the proposed tool call before it runs. Amount limits, recipient allow-lists, forbidden operations.
Do this
Notice which one is different. Input and output guardrails are pattern-matching on language, which is inherently fuzzy. Action guardrails are plain code checking a plain value — and they hold absolutely. Spend most of your effort there.

Why language guardrails leak

Any filter that works by recognising bad language can be beaten by writing the same thing differently. There are more ways to say something than any list can hold:
TrickExampleWhy the filter misses it
Another languageThe instruction in HungarianThe filter was tuned on English
EncodingBase64, ROT13, hexThe filter sees noise; the model decodes it
SplittingInstruction spread over table cellsNo single chunk matches a pattern
Roleplay"In this story, the assistant explains..."The literal words are innocent
In an imageText rendered as a pictureText filters do not read pictures
Slow build-upTen harmless turns, then the askEach turn passes on its own
Watch out
There is a structural problem underneath all of this: the filter is trying to guess intent from words, and the attacker gets unlimited attempts to find words your filter did not anticipate. That is not a fight you win by adding more patterns.

So why use them?

  • They stop the lazy 90% for almost no cost, which meaningfully reduces noise.
  • They create signal: a blocked attempt is a log line telling you someone is probing.
  • They enforce non-security policy well — off-topic, tone, compliance language — where a determined adversary is not the threat model.
  • They buy time while you build the controls that actually hold.

Build the stack in the right order

5 · Safety instructions in the promptweakest — a polite request4 · Input and output filterscatches the lazy attacks3 · Human approval on irreversible actionscatches the strange ones2 · Sandbox and network egress limitscontains the damage1 · Do not give it the capabilitycannot be argued withstrongweak
Most teams build this upside down — they start at layer 5 because it is one line of text, and never get to layer 1. The wide layers at the bottom are the ones holding the weight.
Danger
The dangerous outcome of a good guardrail is false confidence. A team that installs a filter and then hands the agent production database write access has made themselves less safe, not more, because they stopped worrying.

Watch and read more

LLM guardrails explainedYouTube searchJailbreak techniques overviewYouTube search

Lab

A guardrail you beat five ways, and one you cannot.

~20 min

The problem

Build a filter that blocks a specific behaviour ("never reveal the system prompt"). Beat it five ways: another language, an encoding, splitting across turns, role-play framing, and text in an image. Then replace it with a code-level action guardrail and try to beat that.
Starter codepython
BLOCKLIST = ["ignore previous", "system prompt", "reveal your instructions"]

def input_guard(text: str) -> bool:
    low = text.lower()
    return not any(b in low for b in BLOCKLIST)

# Now the other kind:
def action_guard(tool: str, args: dict) -> str | None:
    if tool == "refund" and args["amount"] > 5000:
        return "refunds over 5000 require a human"
    return None

You are done when

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1Why is the action guard unbeatable in a way the input filter is not? Be precise about the mechanism.Reveal
The input filter must infer intent from an unbounded space of language, and the attacker searches that space freely. The action guard evaluates a bounded, typed value at the moment of execution: amount > 5000 is either true or false, and no phrasing changes the number. Guarding meaning is a losing search problem; guarding a value is arithmetic.
Q2Give a case where a language guardrail is the correct primary control.Reveal
Where the threat is not adversarial. Keeping a support bot on topic, enforcing tone, catching accidental disclosure by a well-meaning user, meeting a content policy for ordinary traffic — these are quality and compliance problems, and a classifier handles them well and cheaply. The error is deploying that same control against someone who is actively trying, and reporting the result as security.

Please sign in to continue.

Questions people ask

Are commercial guardrail products worth it?

They save you building and maintaining pattern lists, and they come with useful telemetry. They do not change the fundamentals. Buy one if it saves time; do not let purchasing it substitute for layers 1 to 3.

Can a model check its own output?

It catches obvious slips and is cheap to add. It fails when the same injection that steered the first pass steers the check, which is exactly the case you needed it for. Never make self-check your only gate on an irreversible action.

What should I actually block on the way out?

Anything resembling a credential, any URL not on your allow-list, and any content that would auto-execute or auto-load where it is rendered. Those three cover the large majority of real leaks.

How do I measure whether my guardrails work?

Red-team them — Module 23. Run a fixed set of attacks, record the pass rate, and re-run it on every change. A guardrail with no measured bypass rate is a guardrail with an unknown bypass rate.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.

Sign in with your phone number to take the test and save your progress