Running an AI security programme
Put it all together: inventory, risk tiers, policies, secure development, testing, monitoring, response and reporting, plus a 90-day plan to start.
In 60 seconds
Running an AI security programme
Put it all together: inventory, risk tiers, policies, secure development, testing, monitoring, response and reporting, plus a 90-day plan to start.
The building blocks
| Block | What it means |
|---|---|
| Inventory | A list of every AI system you build, buy or use, including hosted APIs, open models and AI features inside vendor tools. |
| Risk tiers | Group systems by impact: data sensitivity, decisions affected, exposure to untrusted users, autonomy. |
| Policies and standards | Acceptable use, approved models and vendors, data rules, model loading rules, release requirements. |
| Secure AI development life cycle | Threat modelling, data controls, supply chain checks, evaluation gates and approvals built into how teams work. |
| Red teaming and testing | Regular adversarial testing, scaled to the risk tier, using tools and people. |
| Monitoring | Usage, abuse patterns, drift, data leakage signals and access to weights. |
| Incident response | AI scenarios in playbooks: poisoning, model theft, data leakage, harmful output, rollback. |
| Training | Role-based: engineers, data scientists, product owners, leaders and all staff. |
| Metrics and reporting | Clear numbers to leadership and the board, tied to risk. |
Risk tiers drive effort
| Tier | Example | Minimum controls |
|---|---|---|
| Low | Internal summariser on public documents | Inventory entry, approved tool, basic usage policy |
| Medium | Customer-facing chatbot answering from product docs | Threat model, input and output controls, logging with redaction, pre-release testing |
| High | Model that affects credit, hiring or health, or handles sensitive personal data | All of the above plus red teaming, provenance and AI-BOM, human oversight, legal review, regular re-testing |
Metrics that mean something
- Share of AI systems in the inventory that have an owner and a risk tier.
- Share of high-tier systems with a current threat model and red team test.
- Share of models loaded from safe formats with verified hashes or signatures.
- Time to fix serious AI findings, compared with your target.
- AI-related incidents and near misses, with lessons learned.
A 90-day starter plan
- 1
Days 1–30: See
Name an executive sponsor and a programme owner. Build the first AI inventory (ask teams, check expense and cloud records, review vendor tools). Publish a short interim AI use policy. - 2
Days 31–60: Sort
Assign risk tiers. Pick a framework to align with (for example NIST AI RMF). Threat model the top three high-tier systems. Set safe model loading and PII logging rules. - 3
Days 61–90: Act
Add evaluation gates to one pipeline. Run a first red team exercise on one high-tier system. Add AI scenarios to the incident plan and run a tabletop drill. Send the first report to leadership with metrics and next steps.
- Start by buying a tool
- Write a long policy nobody reads
- Block all AI use
- Security works alone
- Start with inventory and owners
- Short, clear rules, with approved options
- Make the safe path the easy path
- Security, ML, legal, privacy and business work together
Watch and read more
Lab
A 90-day AI security plan for your organisation, with an inventory template and board metrics.
The problem
Write a plan for your organisation, or a realistic one such as a mid-sized Indian bank or IT services company.
- Create an inventory template with at least these columns: system, owner, build or buy, model and version, data used, users, risk tier.
- Fill it with at least five real or realistic entries and assign tiers with a one-line reason each.
- Write your 90-day plan in three 30-day phases, with named owners and a clear output for each phase.
- Choose four metrics for your first leadership report and say how you will measure each.
- List the top three risks you expect to find, and what decision you might need from leadership.
You are done when
Hard questions
Try to answer before you reveal. If you can answer these, you understood the lesson.
Q1Teams hide AI use because they fear it will be banned. How do you design the programme to fix this?Reveal
Make disclosure safe and useful. Offer approved tools and quick reviews, so declaring a use case leads to help, not a ban. Keep an amnesty period for the first inventory. Focus controls on high-tier uses and keep low-tier approvals light. When people see the safe path is fast, hidden use falls.
Q2Your metrics show 100% of models have an owner, but an incident still happens. What might the metrics have missed?Reveal
Ownership is a coverage metric, not an effectiveness metric. The owner may not have tested the model, controls may exist on paper only, or the incident may come from a system missing from the inventory. Add metrics on test results, time to fix, inventory completeness checks and near misses, and review incidents to find which assumptions were wrong.
Questions people ask
Who should own the AI security programme?
Usually the security leader owns it, with close partners in ML engineering, privacy, legal, risk and the business. Some companies create an AI governance committee. What matters is one accountable owner and clear roles.
How do we find shadow AI in the inventory?
Ask teams directly and make it safe to answer. Check cloud bills, expense claims, network logs for AI service domains, browser extensions and vendor tools that have added AI features. Repeat regularly: new AI appears all the time.
Do we need a separate AI incident response plan?
Usually no. Extend your existing plan with AI scenarios and people who understand models. Add steps such as rolling back a model version, disabling a feature, rotating keys used by AI systems, and preserving prompts and logs as evidence.
How often should we red team AI systems?
Scale it to the risk tier. Test high-tier systems before launch and after major changes, such as a new model version, new data or new tools. Automated tests can run on every release; deeper human-led exercises can be less frequent.
Lesson test
5 questions. Get 3 right (60%) to pass and complete this lesson.