Apply access control at retrieval time: every query should carry the caller's identity, and results must be filtered by the permissions that identity has on the source documents. Embeddings can leak what they encode, so treat the vector store as a copy of the data it came from, with the same rules.
Why is this easy to get wrong?
A retrieval augmented system turns documents into vectors, stores them, and fetches the closest ones to feed a model. The source system had permissions. The vector store often has none, so anyone who can query it can reach everything indexed. OWASP's entry on vector and embedding weaknesses lists unauthorized access and data leakage from inadequate or misaligned access controls, and cross-context leaks in multi-tenant environments where users or applications share one database.
The model then summarizes whatever it receives. If a restricted document is retrieved for an unauthorized user, the model may repeat it.
What are the design options?
Metadata filtering. Store tenant, owner and permission tags with each vector and require every query to include a filter derived from the authenticated identity. The filter must be added by the server, never by the client or the model.
Separate indexes or collections. Sensitive or per-tenant data lives in its own index with its own credentials. This costs more to run and removes a class of filter bugs.
Permission sync. Update the vector store when source permissions change, or check permissions against the source system at query time on the returned documents. Stale permissions are a common leak.
Pre-retrieval versus post-retrieval checks. Filtering inside the query is better than retrieving and discarding, since the latter wastes work and tends to leak through scores and counts.
What about the embeddings themselves?
OWASP's page also covers embedding inversion, where attackers try to recover source information from vectors, and poisoning, where malicious content is added to the index to steer answers. Treat the store's write path as sensitive: restrict who can add documents, record their origin, and review ingestion of untrusted sources.
A constructed example
A company builds an internal assistant over its wiki. Retrieval returns the top five matching chunks for any employee. A manager asks about reorganization plans and gets a paragraph from a restricted HR page, indexed along with everything else. The fix adds a department and access group tag at indexing time, requires the server to filter by the user's groups on every query, and moves HR pages to a separate index. A weekly test with a low privilege account tries to retrieve restricted content.
What to do
- Inventory every source indexed, with its owner and access rules.
- Carry authenticated identity through to the vector query and derive filters on the server.
- Store permission metadata with each chunk, and sync changes quickly.
- Use separate indexes for tenants or sensitive classes where the risk justifies it.
- Restrict writes to the index, record document origin and review untrusted sources.
- Log queries and retrieved document identifiers for audit, with sensitive text redacted.
- Test with low privilege accounts on a schedule, and fix leaks as incidents.
- Encrypt the store and limit direct database access to the service account.
Safeguard offers guardrails and policies for AI tool use, but retrieval permissions have to be enforced in your own data layer.
Quick check
Can the model decide which documents a user may see?
No. The server must enforce access from authenticated identity, because model instructions can be bypassed.
Are embeddings anonymous?
Do not assume so. OWASP lists embedding inversion as a risk, so treat vectors like the data they came from.
Sources
Last reviewed: 18 October 2026.
