RAG vs CAG: what is the difference and which is safer to run?

Retrieval-augmented generation (RAG) fetches relevant documents at query time and gives them to the model. Cache-augmented generation (CAG) preloads a whole knowledge set into the model's context and caches it, so there is no retrieval step. RAG scales to large, changing data. CAG suits small, stable data and shifts the access control problem.

What is RAG?

The original RAG paper, from 2020, combines a pre-trained language model, the parametric memory, with a non-parametric memory, in that work a dense vector index of Wikipedia accessed by a neural retriever. The authors motivated it by two open problems with language models alone: providing provenance for their decisions and updating their world knowledge. A retrieval step lets the system cite sources and use new documents without retraining.

What is CAG?

A 2024 paper, "Don't Do RAG: When Cache-Augmented Generation is All You Need for Knowledge Tasks," accepted as a short paper at the Web Conference 2025, proposes an alternative for models with long context windows. Its authors say RAG brings retrieval latency, potential errors in document selection and added system complexity. Their method preloads all relevant resources, especially when the documents are of limited and manageable size, into the model's extended context and caches its runtime parameters. At inference the model answers from those preloaded parameters without retrieval. They report that this removes retrieval latency and minimizes retrieval errors while maintaining context relevance, and conclude that for applications with a constrained knowledge base CAG can be a streamlined alternative that achieves comparable or better results with less complexity.

Note the qualifier in their own text: constrained knowledge base. CAG is not presented as a replacement for RAG in general.

How do they differ in practice?

• Knowledge size: RAG can draw on a corpus far larger than any context window. CAG is limited by the window.
• Freshness: RAG sees a new document as soon as it is indexed. CAG must reload and recompute the cache when the content changes.
• Latency: CAG removes the retrieval step. RAG pays for it on every query.
• Failure mode: RAG can retrieve the wrong passages. CAG cannot retrieve wrongly because everything is present, but the model may still attend poorly to one fact among many.
• Provenance: RAG naturally returns the passages it used. CAG has to be asked to cite.

What changes for security?

Access control is the main difference. In RAG you can filter at retrieval: only fetch documents the asking user may see. In CAG everything in the cache is in context for every question answered from that cache, so anyone who can query it can in principle draw out anything in it. If users have different permissions, you need a separate cache per permission group, or CAG is the wrong fit.

OWASP's Top 10 for LLM applications lists vector and embedding weaknesses as a distinct risk, which matters for RAG stores, and also lists sensitive information disclosure and prompt injection. Both approaches share the injection problem: text in the retrieved or preloaded documents can contain instructions the model might follow. Content you load into either path should be treated as untrusted input.

A constructed example

A company has two assistants. The HR policy bot answers from a 60 page handbook that changes twice a year, and everyone in the company may read all of it. CAG fits: load the handbook, cache it, answer fast, refresh on release. The support assistant answers from a million tickets that differ by customer, with access limited by account. RAG fits: retrieve per query and filter by the customer's identity before anything reaches the model. Using CAG for the second would put every customer's tickets in one context.

How to check or apply this

1. Estimate the size of your knowledge set against your model's context window.
2. Decide how often the content changes and how fast changes must appear.
3. Map who may read which content. If permissions differ per user, prefer RAG with retrieval time filtering.
4. For CAG, create one cache per permission group and never mix them.
5. Treat all loaded content as untrusted input, and scan it for injected instructions.
6. Log which documents were retrieved or loaded for each answer.
7. Test with questions that should be refused for a given user.
8. Re-evaluate when your corpus outgrows the context window.

Quick check

Sources

• Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv 2005.11401)
• Chan et al., Don't Do RAG: When Cache-Augmented Generation is All You Need for Knowledge Tasks (arXiv 2412.15605)
• OWASP Gen AI Security Project: Top 10 for LLM Applications

Last reviewed: 18 October 2026.