Personal data in prompts, logs and training data
Personal data leaks into AI systems through prompts, logs and datasets. Learn to minimise it, redact it, limit how long you keep it, and respect people's rights.
In 60 seconds
Personal data in prompts, logs and training data
Personal data leaks into AI systems through prompts, logs and datasets. Learn to minimise it, redact it, limit how long you keep it, and respect people's rights.
Where personal data ends up
- 1
Prompt
The user's message, plus documents and records your app adds. - 2
Model provider
Sent to a hosted model, which may keep it for a period under its terms. - 3
Logs and traces
Debug logs, observability tools and error reports often store full prompts and replies. - 4
Evaluation sets
Real conversations copied into test sets for quality checks. - 5
Training data
Conversations used to fine-tune future models.
Four principles
| Principle | In practice |
|---|---|
| Minimise | Send the model only the fields it needs. Don't add full customer records by default. |
| Redact | Detect and mask personal data before logging, evaluation and training. |
| Limit retention | Keep logs and prompts only as long as needed, then delete automatically. |
| Purpose | Use data only for the purpose people were told about and agreed to. |
A redaction step
[PHONE] so the text stays useful.import re
RULES = [
("[EMAIL]", re.compile(r"[\w.+-]+@[\w-]+\.[\w.]+")),
("[PHONE]", re.compile(r"(?:\+91[\s-]?)?\b[6-9]\d{9}\b")),
("[PAN]", re.compile(r"\b[A-Z]{5}\d{4}[A-Z]\b")),
("[ID12]", re.compile(r"\b\d{4}\s?\d{4}\s?\d{4}\b")),
]
def redact(text: str) -> str:
for tag, rx in RULES:
text = rx.sub(tag, text)
return text
print(redact("Call me on 9876543210 or mail asha@example.com"))Consent, purpose and India's DPDP Act
Handling deletion requests
- Know every place a person's data can be: databases, logs, vector stores, evaluation sets, training sets, backups, and vendors.
- Deleting from databases and logs is straightforward. Removing someone's influence from a trained model is hard; this is one more reason to keep personal data out of training sets.
- Record what you deleted, where, and when.
Vendor data terms
- Does the provider use your prompts to train its models? Can you opt out?
- How long does it keep prompts and outputs, and for what purposes, such as abuse monitoring?
- Where is data processed and stored?
- Which sub-processors see it, and what happens at contract end?
Watch and read more
Lab
A tested PII redaction step with a measured miss rate, and a retention table.
The problem
Use the redaction code from the lesson.
- Write 20 made-up customer messages in the styles your users write (English, Hinglish, a local language in Latin script). Include invented phone numbers, emails, PAN-style numbers and names. Use no real personal data.
- Run the redaction and count what was caught, missed, and wrongly masked.
- Improve at least one rule and run again.
- Write a retention table: for prompts, logs, evaluation sets and training data, say what is kept, how long, and who can access it.
You are done when
Hard questions
Try to answer before you reveal. If you can answer these, you understood the lesson.
Q1Your redaction catches all phone numbers in tests but names are missed. What is your plan?Reveal
Names have no fixed pattern, so regular expressions cannot catch them well. Add a named-entity detector suited to Indian names and languages, and test it on realistic samples. Accept that it will miss some, so also restrict access to raw logs, shorten retention, and avoid training on raw conversations.
Q2Why is "we have consent" not the end of the privacy discussion?Reveal
Consent covers a specific purpose, and people can withdraw it. You still need to minimise data, keep it secure, delete it when the purpose ends, and honour rights requests. Using data for a new purpose, such as training a new model, may need fresh notice or consent. Security failures are still breaches, even with consent.
Questions people ask
Should we log full prompts for debugging?
Log redacted prompts by default. If full prompts are truly needed for a short investigation, collect them for a limited time, with restricted access and automatic deletion. Never let full prompts flow into general-purpose log tools by accident.
Can we train on customer chats if we remove names?
Only if your notice and consent cover that purpose, and after careful redaction. Remember that removing names is pseudonymisation at best. Many teams choose to train only on synthetic or specially collected data to avoid this risk.
What about employees pasting customer data into public chatbots?
That is a data leak risk to handle with policy, approved tools, training and, where possible, technical controls. Give people a safe, approved option so they are not tempted to use personal accounts.
Is this legal advice?
No. This lesson gives general principles. Laws and rules change and depend on your facts. Work with your legal and privacy teams on your actual obligations.
Lesson test
5 questions. Get 3 right (60%) to pass and complete this lesson.