AI for security defendersAdvancedLesson 1625 min read

Detection engineering with AI help

Detection engineers turn attacker behaviour into rules. AI can draft and translate rules fast, but every rule must be tested and reviewed like code.

✦ Ask the tutor🧪 Lab · 35 min📝 Lesson test
Lesson in motion

In 60 seconds

Detection engineering with AI help

Detection engineers turn attacker behaviour into rules. AI can draft and translate rules fast, but every rule must be tested and reviewed like code.

1/5
In simple words
A smoke alarm is a rule: "If there is smoke, beep." A good alarm beeps for fire, but not every time you make toast. Detection engineers build alarms for computers, and they test them so they do not beep all day for nothing.
A detection rule describes a pattern in logs that should raise an alert. Sigma is an open, generic rule format written in YAML. A Sigma rule can be converted into queries for many SIEM tools, so teams can share detections.

A simple Sigma rule

This rule looks for PowerShell started with an encoded command. Attackers often use encoding to hide what a script does, but some admin tools use it too.
Sigma rule: encoded PowerShell command lineyaml
title: PowerShell Started With Encoded Command
id: 6f1c2a54-3b8e-4d7a-9c61-2e5f8a0b7d13
status: experimental
description: Detects PowerShell run with an encoded command argument, often used to hide script content.
author: OmniLingo learner
tags:
    - attack.execution
    - attack.t1059.001
logsource:
    category: process_creation
    product: windows
detection:
    selection_img:
        Image|endswith:
            - '\powershell.exe'
            - '\pwsh.exe'
    selection_cli:
        CommandLine|contains:
            - ' -enc'
            - ' -ec '
    condition: selection_img and selection_cli
falsepositives:
    - Some legitimate software deployment and management tools
level: medium
Read it from top to bottom: which logs (logsource), what to match (detection), when to fire (condition), and known harmless causes (falsepositives). Note that -enc also matches -EncodedCommand, because Sigma's contains is case-insensitive by default.

Where AI helps

  • Drafting a first rule from a threat report or a hunt query
  • Translating a rule from one query language to another
  • Explaining an existing rule to a new team member
  • Suggesting likely false positives and test cases
Watch out
AI can produce rules with wrong field names, invalid modifiers, or wrong ATT&CK IDs. It can also make rules too broad (alert storms) or too narrow (easy to bypass by changing one character). Validate with Sigma tooling and test on real samples.

Test against sample logs

  1. 1

    Positive test

    A sample event the rule must catch. Does it fire?
  2. 2

    Negative test

    Normal events it must not catch. Does it stay quiet?
  3. 3

    Bypass test

    A small change an attacker could make. Does it still fire?
  4. 4

    Backtest

    Run on recent real data to count how often it would fire.

False-positive tuning

If a rule fires too often on harmless activity, analysts start ignoring it. Tune carefully: add narrow exclusions (a specific signed tool, a specific parent process), not broad ones like "ignore all admins". Every exclusion is a place an attacker could hide, so document why it exists.

Detection as code

Rules as clicks
  • Rules edited directly in the SIEM console
  • No history of who changed what
  • No tests before going live
Detection as code
  • Rules stored in version control (like Git)
  • Every change reviewed by another person, AI drafts included
  • Automatic checks and tests run before a rule is deployed
Note
An AI-drafted rule should go through exactly the same review and tests as a human-written one. "The AI wrote it" is not a review.
Try this. Ask an AI to explain each line of the Sigma rule above. Did it explain the condition line correctly?

Watch and read more

SigmaHQ: Sigma rule repository and specificationSigmaHQ · GitHub · reading

Lab

A Sigma rule tested against sample events, with a note on false positives and bypasses.

~35 min

The problem

Use the sample process events below (fictional).

  1. Ask an AI to write a Sigma rule that catches PowerShell with an encoded command. Compare it with the rule in this lesson.
  2. By hand, go through each sample event and mark whether your rule would fire. (If you can, install the Sigma tooling and check the rule is valid.)
  3. Identify which events are true positives, false positives and missed attacks.
  4. Propose one tuning change and one improvement against bypasses. Explain the trade-off.
Starter codetext
1. Image=C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
   CommandLine=powershell.exe -NoProfile -enc SQBFAFgAIAAoA...
   ParentImage=C:\Program Files\Microsoft Office\root\Office16\WINWORD.EXE
2. Image=C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
   CommandLine=powershell.exe -File C:\Scripts\daily_report.ps1
   ParentImage=C:\Windows\System32\svchost.exe
3. Image=C:\Program Files\PowerShell\7\pwsh.exe
   CommandLine=pwsh.exe -EncodedCommand JABzAD0A...
   ParentImage=C:\Program Files\ExampleDeployTool\agent.exe
4. Image=C:\Users\Public\ps.exe
   CommandLine=ps.exe -enc SQBFAFgA...
   ParentImage=C:\Windows\explorer.exe

You are done when

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1Event 4 is a renamed copy of PowerShell. How could a rule catch it without matching the file name?Reveal

Use a field that does not change when the file is renamed. Many process logs record the original file name from the program's version information, which stays the same after a rename. A rule can match on that field, or on the encoded-command pattern alone for any process. Check which fields your log source really provides.

Q2Why is the Word document starting PowerShell (event 1) more worrying than the deployment tool (event 3)?Reveal

Office programs rarely have a good reason to start PowerShell with hidden commands. It is a common sign of a malicious document. A known deployment tool doing it may be normal for your company. The parent process gives context, which is why good rules and good triage look at process chains, not single events.

Please sign in to continue.

Questions people ask

Why write Sigma instead of writing directly in my SIEM's language?

Sigma is tool-neutral, so a rule can be shared and converted for different SIEMs. It is also easy to store and review in version control. Many teams still write some rules directly when they need features Sigma does not cover.

How do I check that a Sigma rule is valid?

Use the official Sigma tooling (for example, the sigma-cli converter and validators from the SigmaHQ project) to check the format and convert it. Then test the converted query on sample events in your own tool.

Can AI translate an SPL query into KQL correctly?

Often it gets close, but functions, field names and time handling differ between languages. Test the translated query on the same data and compare the results with the original.

What makes a detection easy to bypass?

Matching one exact string or file name. Attackers can rename files or change letters. Stronger rules look at behaviour, like which process started which, and combine several signals.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.

Sign in to take the test and save your progress