Agent securityMiddleLesson 245 min read

When it goes wrong: response and kill switches

Assume an incident. Write the plan now, while nothing is on fire and you can still think clearly.

🧪 Lab · 15 min📝 Lesson test
Lesson in motion

In 60 seconds

When it goes wrong: response and kill switches

Assume an incident. Write the plan now, while nothing is on fire and you can still think clearly.

1/6
In simple words
Every machine needs a big red button that stops it instantly. Know where the button is before you need it.
Agent incidents are different from ordinary outages in three ways, and each one changes how you respond:
  • It is still running. Unlike a data breach discovered months later, a misbehaving agent is often mid-run, doing more damage while you read the alert.
  • The damage is distributed. Not one bad request — two hundred small reasonable-looking actions across several systems.
  • The cause is textual. There is no malformed packet to point at. Somewhere, a sentence changed the plan.

The first ten minutes

  1. 1

    Stop it

    Kill the run. Not a graceful shutdown — an immediate halt. If you cannot do this in one action, that is your top priority after the incident.
  2. 2

    Revoke its access

    Invalidate the agent's tokens. This is why per-agent identity from Module 21 matters: you revoke one thing, not everything.
  3. 3

    Freeze the evidence

    Snapshot logs, context, memory store and tool history before anything rotates or expires.
  4. 4

    Work out what it touched

    Use the correlation ID to list every action in the run. Then check whether earlier runs did the same thing — the alert is rarely the first occurrence.
  5. 5

    Contain the spread

    If long-term memory or a cache may be poisoned, quarantine it. If other agents consumed its output, trace them too.
  6. 6

    Then diagnose

    Only now ask why. Read the context immediately before the first bad action; the cause is almost always right there.
Danger
Do not "just restart it and watch". If the agent's memory or knowledge base is poisoned, restarting reproduces the attack with a clean-looking log and destroys evidence.

Build the kill switch before you need it

ControlWhat good looks like
Global stopOne command, stops every agent run in flight, in under a second
Per-agent stopDisable one agent without touching the others
Per-tool disableTurn off send_email system-wide while keeping the agent useful
Token revocationOne action invalidates that agent's credentials everywhere
Feature flag rollbackReturn to the previous prompt and tool set without a deploy
Circuit breakersAutomatic halt on volume, spend or error thresholds — no human needed
Do this
Test the kill switch on a schedule, like a fire drill. A stop button nobody has pressed in six months is a stop button that does not work.

The post-incident questions that matter

  1. What text caused it, and how did that text reach the agent?
  2. Which layer of defence should have caught it, and why did it not?
  3. Could a capability be removed so this class of attack becomes impossible?
  4. How long did detection take, and what would have shortened it?
  5. What regression test now exists so this cannot silently return?
Note what is missing from that list: blame. The useful output of an incident review is a changed system, not a changed person.

Watch and read more

Incident response fundamentalsYouTube searchBlameless postmortemsYouTube search

Lab

A kill switch you have actually pressed, timed.

~15 min

The problem

Implement three controls: stop every run in flight, revoke one agent's credentials, and disable one tool globally without a deploy. Then run a fire drill — start a long-running agent and time how long each takes from decision to effect.
Starter codepython
# Tool-level circuit breaker, checked by the executor before every call
def tool_enabled(name: str) -> bool:
    return not redis.sismember("agent:disabled_tools", name)

def emergency_stop(run_id: str | None = None):
    redis.set("agent:halt", "1" if run_id is None else "")
    if run_id:
        redis.sadd("agent:halted_runs", run_id)
    # executors check this before EVERY tool call, not just between steps

You are done when

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1Your stop flag is checked between steps. Why is that not enough, and what is the fix?Reveal
A single step can be a long-running tool call — a large query, a slow API, a batch write. Between-step checks mean your stop takes effect after the damage. Check before every tool call and pass a cancellation token into the tool itself so in-flight work aborts. And revoke the credential at the same moment: revocation stops work that has already left your process, which a flag cannot.
Q2An agent may be poisoned. A colleague says 'restart it and watch carefully.' Give the strongest argument against.Reveal
If the poisoning lives in long-term memory, a cache or the knowledge base, restarting reproduces the attack from a clean-looking start, destroys volatile evidence, and hands you a second incident that looks like a first. Correct order: halt, revoke, snapshot logs and memory, determine what it touched, quarantine the suspect store, and only then bring anything back. 'Restart and watch' is the reflex from stateless outages and it is wrong here.

Please sign in to continue.

Questions people ask

Should agents stop automatically when something looks wrong?

Yes, on clear numeric signals: spend over budget, action volume over threshold, repeated tool failures, a canary token seen leaving. Automatic halts are cheap and they act at machine speed, which is the speed the problem is moving at.

How do we undo damage?

Design for it in advance: soft deletes, transaction logs, reversible operations, and staged writes. If an action genuinely cannot be undone, it belongs behind a human approval — that is what puts it in that category.

Do we need to tell customers?

If personal data was exposed, you likely have a legal obligation with a deadline measured in hours or days depending on jurisdiction. Involve legal early. Do not let the technical investigation delay the disclosure clock.

How do we know it was an attack and not a bug?

Often you will not, quickly, and the response is identical either way: stop, contain, investigate. Look for the fingerprint — did the behaviour change immediately after ingesting a particular piece of external content? That is your tell.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.

Sign in with your phone number to take the test and save your progress