What AI can and cannot do for defenders
AI is a fast helper for reading, writing and explaining. It is not a source of truth. Learn where the line is, and how to handle data safely.
In 60 seconds
What AI can and cannot do for defenders
AI is a fast helper for reading, writing and explaining. It is not a source of truth. Learn where the line is, and how to handle data safely.
Where AI helps
| Task | How AI helps | What you still check |
|---|---|---|
| Summarising | Turns a long alert, ticket history or report into a short summary | That nothing important was dropped or changed |
| Explaining | Explains a command line, a script or an unfamiliar log field | The explanation against documentation or a test |
| Drafting queries | Writes a first draft of a log search or detection rule | That it runs, and finds what it should on known data |
| Drafting reports | Writes incident notes, timelines and management summaries | Every time, name, number and claim against the evidence |
| Translating | Translates phishing emails or notes from other languages; converts between query languages | Key meaning with a second method or a colleague |
Where AI is weak
- Ground truth. It does not know what really happened on your network. Only your logs and tools know that.
- Made-up indicators. It can invent IP addresses, domains, file hashes or malware family names that look real.
- CVE details. It can mix up vulnerability IDs, affected versions and fixes. Always check the official advisory.
- Recent events. Its training has a cut-off date, so it may not know new threats unless a tool gives it current data.
- Confidence. It sounds equally sure when it is right and when it is wrong.
Safe data handling
- Pasting a full customer database export to "find the odd rows"
- Pasting a config file that contains a real API key
- Using a personal chatbot account for company incidents
- Using the company-approved AI tool, with its data rules
- Removing or replacing secrets and personal data before sharing
- Sharing only the few log lines needed, with names masked when possible
A human signs off
- 1
Give evidence
Paste or connect the real logs and alert data (in an approved tool). - 2
Ask a clear task
Summarise, explain, draft. Ask it to point to the evidence for each claim. - 3
Verify
Check indicators, CVEs and conclusions against your tools and official sources. - 4
Decide and record
A person makes the call and writes down why.
netstat -ano. Then check its answer against the official documentation. Did it get every flag right?Watch and read more
Lab
A personal "AI in the SOC" rule card: what you will use AI for, what you will always check, and what you will never paste.
The problem
Write a one-page rule card for yourself or your team.
- List 5 SOC tasks where you would use AI (for example: summarise a ticket, explain a command line).
- For each one, write what you must check before trusting the result.
- List at least 6 types of data you will never paste into a public chatbot.
- Test one safe task: ask a chatbot to explain a harmless command such as
whoami /groupsorps aux. Compare the answer with the official documentation and note any mistakes.
You are done when
Hard questions
Try to answer before you reveal. If you can answer these, you understood the lesson.
Q1Why is "ask the AI to cite its sources" not enough to stop invented indicators?Reveal
A model can also invent sources, or cite a real source that does not say what it claims. Citations help only if you open them and check. For indicators, the only reliable source is your own telemetry or a trusted threat feed that you query directly.
Q2Your AI tool gives a correct answer 9 times out of 10 on a task. Is that good enough to let it close alerts alone?Reveal
It depends on the cost of the 1 in 10. For low-risk, easily reversed work, it may be acceptable with sampling and review. For high-severity alerts, one wrong closure can mean a missed breach. So the decision depends on severity and impact, not just the accuracy number, and high-severity decisions should keep a human reviewer.
Questions people ask
Will AI replace SOC analysts?
AI changes the work more than it removes it. Repetitive reading and writing gets faster. Judgement, investigation, talking to people and taking responsibility still need humans. Analysts who can use AI well and check it carefully become more valuable.
Why would a model invent an IP address or hash?
It has seen millions of IP addresses and hashes in text, so it knows what they look like. When asked for one it does not have, it can produce a realistic-looking one. It is not lying on purpose; it is predicting likely text. That is why every indicator must come from your data or a trusted feed.
Is a company AI tool automatically safe for any data?
No. Approved tools come with rules: which data types are allowed, where data is stored, how long it is kept. Read and follow those rules. Data protection laws, such as India's Digital Personal Data Protection Act 2023, and your contracts with customers still apply.
Can AI read a malware report and tell me if we are affected?
It can pull out the indicators and techniques from the report and draft searches for them. Whether you are affected depends on running those searches in your own tools and checking the results yourself.
Lesson test
5 questions. Get 3 right (60%) to pass and complete this lesson.