Incident response for agentic systems
The earlier lesson covered the first ten minutes. This one turns it into a company playbook: roles, steps, evidence, clean-up and a tabletop exercise to practise it.
In 60 seconds
Incident response for agentic systems
The earlier lesson covered the first ten minutes. This one turns it into a company playbook: roles, steps, evidence, clean-up and a tabletop exercise to practise it.
The playbook
- 1
Detect
Alerts from the gateway and logs (first-time tool, spike, new destination, DLP hit), user reports, or a partner telling you. - 2
Contain
Use the kill switch for the agent or the tool. Revoke its tokens and OAuth grants. Disable risky tools for all agents if the cause is unclear. - 3
Preserve evidence
Snapshot tool-call logs, conversation context, memory stores, retrieved documents and configuration before anything expires. - 4
Investigate
Use correlation IDs to list every action. Find the content read just before the first bad action. Check earlier runs and other agents that used the same source. - 5
Eradicate
Remove poisoned memory entries, documents, tickets or tool descriptions. Fix the gap in policy or permissions. - 6
Recover
Restore from clean state, rotate exposed secrets, re-enable with tighter limits, and watch closely. - 7
Learn
Blameless review: what happened, why controls missed it, what changes. Add tests and alerts.
Who does what
| Role | Responsibility |
|---|---|
| Incident lead | Runs the response, makes containment calls, keeps the timeline |
| Agent owner | Knows what the agent is for, what it can reach, and what "normal" looks like |
| Platform / identity team | Pulls the kill switch, revokes tokens and grants, preserves logs |
| Data protection / legal | Decides whether personal data was affected and what must be reported, and to whom |
| Communications | Informs affected users, customers or partners with clear, accurate messages |
Evidence that only agent systems have
- Tool-call logs with arguments, results and the acting user.
- The full context of the run: system prompt version, user messages, retrieved documents, tool outputs.
- Memory and vector store contents, with when and from where each entry was added.
- Tool descriptions and server versions at the time of the incident (they may have changed since).
- Approval records: who approved what, and what they were shown.
Reporting duties
Tabletop exercise
Watch and read more
Lab
A completed tabletop exercise with a timeline, decisions and a list of fixes.
The problem
Run a tabletop exercise alone or with 2–4 colleagues or classmates, using the HR helpdesk scenario from the lesson (or your own made-up one). Do not touch any real system.
- Assign roles: incident lead, agent owner, platform team, data protection, communications.
- Walk through each playbook step. For each, write the decision, who makes it, and how long it would realistically take.
- Add the two "injects" in the starter at the times shown and decide how you respond.
- List every gap you found (missing logs, unclear owner, no kill switch, and so on).
- Write at least five fixes, each with an owner and a date.
TABLETOP: HR helpdesk agent gives staff an unknown link
T+0 min Three employees report the agent told them to "verify your salary account" at a link.
T+10 min INJECT 1: A fourth employee says they entered their details on that page.
T+30 min INJECT 2: The wiki history shows a contractor account edited the payroll FAQ page on Friday.
The same wiki is used by the IT helpdesk agent.
Questions to answer at each step:
- What do we switch off, and how?
- What evidence do we preserve, and where is it?
- Who else must know (inside and outside the company)?
- What do we tell staff, and when?You are done when
Hard questions
Try to answer before you reveal. If you can answer these, you understood the lesson.
Q1Inject 1 says an employee entered their details. How does that change your response?Reveal
It is now also a credential or personal data incident, not only a content problem. Help the employee secure their accounts (password reset, bank contact if needed), check whether other people entered details, involve the data protection team about reporting duties, and consider reporting the phishing site. In India, people can also be pointed to cybercrime.gov.in or the 1930 helpline if money is at risk.
Q2Why is the contractor's account a key lead, and what must you avoid assuming?Reveal
It shows who or what changed the page, so it points to the entry point: the account may be compromised, misused, or used by an automated tool. Do not assume the contractor is guilty; accounts are often taken over. Disable or reset the account, review its other activity, and investigate with the facts before drawing conclusions.
Questions people ask
Should we restart the agent after stopping it, to see if it happens again?
Not before you preserve evidence and check for poisoned memory, documents or tool descriptions. A restart can repeat the harm and overwrite useful state. Investigate first, then re-enable with tighter limits.
How is this different from a normal security incident?
The process is the same shape. The differences are the evidence (context, memory, tool calls), the cause (often text, not code), and the speed (agents act fast and at scale). Your normal team should run it, with the agent owner added.
How often should we run tabletop exercises?
Regularly, and whenever you launch a new high-risk agent. Many teams fold agent scenarios into their existing exercise schedule. Each exercise should end with a short list of fixes and owners.
What should we tell affected users?
What happened in plain words, what data or actions were affected, what you have done, and what they should do (for example, ignore a message, reset a password). Be accurate and avoid guessing. Your legal and communications teams should review it.
Lesson test
5 questions. Get 3 right (60%) to pass and complete this lesson.