To secure a RAG application, enforce the user's permissions at retrieval time, validate and authenticate every document before it is embedded, separate tenants in the vector store, treat retrieved text as untrusted input, give the model only the tools it needs, and log every retrieval. OWASP lists these risks as LLM08.
Key takeaways
- Retrieval is an access control decision. The vector store must know who is asking.
- Anything that enters the knowledge base can later steer the model.
- Retrieved text is untrusted, even when it came from your own wiki.
- Multi-tenant stores need hard partitions, not a filter you hope is applied.
- Logs of what was retrieved are your evidence when something goes wrong.
What is the main risk in a RAG application?
The main risk is that the model can read and repeat anything the retriever hands it. OWASP LLM08:2025, Vector and Embedding Weaknesses, says flaws in how embeddings are created, stored or retrieved can let attackers inject harmful content, skew outputs or expose sensitive data.
Its listed risks include unauthorized access and data leakage, cross-context leaks in multi-tenant databases, embedding inversion, and data poisoning.
How do you stop users reading documents they should not?
Use a permission-aware vector store and check the caller's rights on every query. OWASP recommends fine-grained access controls and partitioning data logically and by access level, so groups can only retrieve their own information.
In practice, store an access label or group list with each chunk, then filter by the authenticated user on the server, never in the prompt. Do not let the model decide who may see what.
How do you keep poisoned documents out?
Validate and authenticate sources before ingestion, and audit the knowledge base for hidden content. OWASP gives an example of a résumé with white-on-white text that tries to steer a RAG based screening system. Its suggested mitigation is text extraction that detects hidden content, plus validating documents before they enter the knowledge base.
Accept data only from verified sources, and record where each chunk came from.
Should retrieved text be trusted?
No. Treat retrieved chunks as untrusted input, since indirect prompt injection arrives inside documents. NeMo Guardrails documents retrieval rails that apply to retrieved chunks in RAG and can reject or alter them, which is one way to add that check.
How do you limit what a hijacked model can do?
Reduce tools and permissions. OWASP LLM06:2025, Excessive Agency, names excessive functionality, excessive permissions and excessive autonomy as root causes. Its prevention advice includes narrow tools over open-ended ones, least-privilege database and API access, scoped access tied to the individual user, and human approval for high-impact actions such as sending or deleting.
What should you log?
Keep detailed, tamper-resistant logs of retrieval activity. OWASP lists this to detect and respond to suspicious behavior. Log the user, the query, the chunk IDs returned and their sources.
A checklist for your RAG pipeline
- Each chunk carries an access label and a source.
- Retrieval filters by the authenticated user on the server.
- Tenants use separate partitions or collections.
- Documents are validated, and hidden text is detected, before embedding.
- Retrieved text goes through an input check before the model sees it.
- The model has only the tools the task needs, with scoped credentials.
- Retrievals are logged and reviewed.
Also scan the packages your pipeline imports. The free scan at gold.safeguard.sh/scan runs in your browser, needs no login, and accepts requirements.txt or package.json with up to 100 dependencies. For the tool layer, see the MCP guardrail demo.
Frequently asked questions
Is RAG safer than fine-tuning on private data?
It can be easier to control, since you can enforce access at retrieval time. It still leaks data if the vector store has no permission checks.
Can embeddings leak the original text?
OWASP lists embedding inversion, where attackers may reconstruct much of the source text from embeddings. Treat embeddings as sensitive as the source.
How do I test my RAG app?
Run adversarial tests such as DeepTeam, which lists RAG pipelines among its targets, and add your own cross-user retrieval checks.
Does filtering in the prompt stop cross-tenant leaks?
No. Enforce the filter in the retrieval query on the server, because a prompt instruction can be overridden by injected text.
Sources
- OWASP LLM08:2025 Vector and Embedding Weaknesses
- OWASP LLM06:2025 Excessive Agency
- NeMo Guardrails on GitHub
- DeepTeam on GitHub
Last reviewed: 19 October 2026.
