Securing AI applicationsMiddleLesson 1396 min read

Securing RAG: permissions for retrieved documents

Retrieval decides what the model sees. If retrieval ignores who the user is, the model will happily answer from documents they should never read.

✦ Ask the tutor 🧪 Lab · 30 min 📝 Lesson test

Speed

Lesson in motion

In 60 seconds

Securing RAG: permissions for retrieved documents

Retrieval decides what the model sees. If retrieval ignores who the user is, the model will happily answer from documents they should never read.

1/5

In simple words

A librarian fetches books for you. If you ask about school rules and they bring the principal's private diary because it matched your words, the problem is the librarian, not your question.

RAG (retrieval-augmented generation) searches your documents, puts the best matching chunks into the prompt, and lets the model answer from them. The model uses whatever it is given. It does not know who is allowed to read what. So the permission check must happen at retrieval, before the model sees the text.

The core rule: filter at query time

1
User signs in
Your app knows who they are from the session: user id, tenant, groups.
2
Build the filter in code
Tenant and access groups come from the session, never from the model or the chat.
3
Search with the filter
The vector or keyword search only considers chunks this user may read.
4
Rank and send
Only permitted chunks reach the model, with their source ids for citation.
5
Log the sources
Record which document ids were used for each answer.

Python: permission filter inside the search (generic)python

def retrieve(query_vector, session, k=5):
    user = session.user                      # from login
    return vector_db.search(
        vector=query_vector,
        top_k=k,
        # Filter syntax differs between databases. The idea does not:
        # tenant is always required, and groups come from the session.
        filter={
            "tenant_id": user.tenant_id,
            "allowed_groups": {"any_of": user.groups},
        },
    )

Common mistakes

Telling the model "only use documents this user may see"
One shared index for all customers, filtered "when needed"
Copying permissions at indexing time and never updating them
An answer cache keyed only on the question text

Better

Filtering in the search query, enforced by code
Tenant filter on every query, or a separate index per tenant
Syncing permission changes and removals quickly
Cache keys that include the user's permission scope, or no shared cache

Multi-tenant separation

An IT services company may build one knowledge assistant for several client projects. Client A's documents must never appear in an answer for client B. You have a few options:

Approach	Strength	Cost
Separate index or database per tenant	Strongest separation; a filter bug cannot mix tenants	More to run and pay for
Separate namespace or collection per tenant	Good separation if access keys are also separate	Depends on the product's features
Shared index with a mandatory tenant filter	Cheapest and simplest to run	One missing filter leaks everything; needs strong tests

Poisoned documents

Anyone who can write into an indexed source can plant instructions: a wiki page, an uploaded PDF, a support ticket, a product review, a crawled website. When that chunk is retrieved, it becomes indirect prompt injection, and the planted content is also a form of data poisoning.

Know every source you index, and who can write to it.
Keep untrusted public content (reviews, uploads) separate from trusted policy documents, and label trust level per chunk.
Limit what the model can do after reading retrieved text: fewer tools, approvals for risky actions.
Review changes to high-trust sources, and watch for hidden text such as white-on-white or tiny fonts in uploads.

Citations and freshness

Show which documents an answer came from, so users can check. Store a document id, version and date with every chunk. When a document is deleted, replaced or expires, remove its chunks quickly. Otherwise last year's leave policy keeps answering questions long after HR replaced it.

Watch out

Answers can leak even when the document is not shown. A title, a summary, or "I found 3 documents about the layoff plan but cannot show them" already tells the user something. Filter first, so forbidden documents are never part of the search results at all.

Try this. In a RAG app you know, find where the search query is built. Does the tenant or user id come from the session? What happens if that field is empty?

Watch and read more

Lab

A toy retrieval function with a working permission filter and passing tests.

~30 min

The problem

On your own computer, run the starter code. It is a toy "search" over five fake documents with simple word matching. Right now it returns documents to anyone.

Change retrieve so it only considers documents with the user's tenant and at least one matching group.
Make it fail closed: if the user has no tenant, raise an error.
Run the tests until they all pass.
Add one test of your own: a user in the right tenant but the wrong group.

Practice lab

Edit the code and run it. Python runs in your browser — nothing is sent to our server.

Ctrl + Enter to run

DOCS = [
    {"id": 1, "tenant": "acme",  "groups": {"all"}, "text": "leave policy 24 days per year"},
    {"id": 2, "tenant": "acme",  "groups": {"hr"},  "text": "salary bands for 2025 review"},
    {"id": 3, "tenant": "zenco", "groups": {"all"}, "text": "leave policy 18 days per year"},
    {"id": 4, "tenant": "acme",  "groups": {"eng"}, "text": "deploy guide for the billing service"},
    {"id": 5, "tenant": "zenco", "groups": {"hr"},  "text": "salary bands and bonus plan"},
]

def score(query, text):
    return len(set(query.lower().split()) & set(text.split()))

def retrieve(query, user, k=3):
    # TODO: filter by user["tenant"] and user["groups"] BEFORE scoring.
    # TODO: if user has no tenant, raise PermissionError.
    ranked = sorted(DOCS, key=lambda d: score(query, d["text"]), reverse=True)
    return [d["id"] for d in ranked if score(query, d["text"]) > 0][:k]

asha = {"tenant": "acme", "groups": {"all", "eng"}}
meera = {"tenant": "acme", "groups": {"all", "hr"}}
ravi = {"tenant": "zenco", "groups": {"all"}}

assert retrieve("leave policy", asha) == [1]
assert retrieve("salary bands", asha) == []
assert retrieve("salary bands", meera) == [2]
assert retrieve("leave policy", ravi) == [3]
try:
    retrieve("leave", {"tenant": None, "groups": {"all"}})
    raise AssertionError("should fail closed")
except PermissionError:
    pass
print("all tests passed")

Output

No output. Add print(...) to see values.

You are done when

All given tests pass without changing them
The filter is applied before scoring, not after
A missing tenant raises an error instead of searching everything
Your extra test covers right tenant, wrong group

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1Your filter works in the chat path. Name two other code paths in a RAG product where the same filter is often forgotten.Reveal

Common ones: background jobs that summarise or tag documents, "related documents" or autocomplete features, answer caches, analytics dashboards that show popular questions and answers, admin or debug search tools, and evaluation scripts that copy production data. Every path that reads the index needs the same filter, ideally through one shared function.

Q2Why is a separate index per tenant sometimes worth the extra cost?Reveal

Because it turns a logic rule into a physical boundary. With a shared index, one missing or wrong filter in any code path can mix tenants. With separate indexes and separate credentials, that bug cannot reach another client's data. For clients with strict contracts or sensitive data, this safety can matter more than cost.

Please sign in to continue.

Questions people ask

Can't we filter the results after retrieval instead?

You can, if nothing else sees the dropped chunks. But post-filtering often returns too few results and is easy to forget in one code path, such as a summary job or a cache. Filtering inside the query is safer and gives better answers.

Does fine-tuning a model on our documents avoid this problem?

No, it makes it worse. Knowledge baked into model weights cannot be filtered per user. Anyone who can use the model may be able to extract it. Keep permissioned data in retrieval, where you can filter it.

What if the empty tenant filter matches everything?

That is a real and common bug. Make the code fail closed: if the tenant or groups are missing, refuse to search. Write a test for exactly this case.

How fast must permission changes reach the index?

As fast as the risk demands. When someone leaves a project, their access to its documents should end quickly. Many teams sync permissions on a schedule and also on important events like removal from a group.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.

Securing RAG: permissions for retrieved documents

Securing RAG: permissions for retrieved documents

The core rule: filter at query time

User signs in

Build the filter in code

Search with the filter

Rank and send

Log the sources

Multi-tenant separation

Poisoned documents

Citations and freshness

Watch and read more

The problem

Practice lab

You are done when

Hard questions

Questions people ask