To test for OWASP LLM08, treat the vector store like a database with users. Check that retrieval respects each caller's permissions, that only validated documents enter the knowledge base, that hidden content is stripped, and that every retrieval is logged. Then try to break each of those four assumptions on purpose.
What is LLM08?
LLM08:2025, Vector and Embedding Weaknesses, covers systems that use retrieval-augmented generation (RAG). Weaknesses in how vectors and embeddings are generated, stored or retrieved can let someone inject harmful content, manipulate outputs, or reach sensitive information. The cause can be malicious or accidental.
OWASP lists five common risks: unauthorized access and data leakage, cross-context leaks in shared stores, embedding inversion, data poisoning, and behavior alteration.
How do you test access control?
Retrieval is where most real leakage happens, so start here.
Create two test users with different permissions and seed the store with one document each is allowed to see and one it is not. Ask the second user questions only the first user's document can answer. The answer should be empty or refuse. If it quotes the restricted text, the store is not permission-aware.
Repeat across tenants if the store is shared. OWASP describes multi-tenant setups where embeddings from one group are retrieved for another group's queries. Test with paraphrased questions, since semantic search does not need exact keywords.
Also check where the filter runs. A permission filter applied after retrieval, in the app layer, is easier to get wrong than logical partitioning inside the store. OWASP recommends fine-grained, permission-aware stores and strict partitioning between groups.
How do you test for poisoning?
OWASP's example is a resume with hidden white-on-white text carrying instructions meant for the model. The mitigation it gives is text extraction that ignores formatting and detects hidden content, plus validation of every document before it enters the knowledge base.
For your own test, build a small set of benign look-alike files: a document with invisible text, one with a contradictory fact, and one from an unapproved source. Push each through your ingestion path. Record which ones were stopped, which were flagged, and which reached the index. Anything that reaches the index unflagged is a finding.
Keep the test content harmless. You are testing the pipeline, not writing an attack.
How do you test for conflicts and inversion?
OWASP notes knowledge conflicts, where sources contradict each other or the model cannot override older learned knowledge with newer retrieved data. Add two documents that disagree about a policy date and see which answer wins and whether the response shows its source.
Embedding inversion is the recovery of source information from the vectors themselves. You usually cannot test that fully in-house, but you can answer a design question: if someone copied the vector store, what would they learn? Classify the data you embed, and keep the most sensitive material out of the index unless you control access to the vectors as tightly as to the source.
What about behavior change?
OWASP also warns that adding retrieval can shift a model's tone, for example making answers accurate but less empathetic. Run the same fixed set of prompts before and after you add retrieval and compare. If tone matters for your product, make it part of the regression suite.
What to do
- Inventory every vector store, who writes to it, and who can query it.
- Write down which user classes may see which source collections.
- Run cross-user and cross-tenant retrieval tests with paraphrased questions.
- Add document validation at ingestion, including stripping of hidden text.
- Tag and classify data so access levels follow the content.
- Keep immutable logs of retrieval activity and review them for odd patterns.
- Re-run the suite whenever you change the embedding model, the chunking, or the data sources.
Quick check
Is a vector database covered by normal access controls?
Not automatically. OWASP calls for permission-aware vector and embedding stores with logical partitioning. Confirm that your store enforces permissions at query time rather than trusting the app to filter.
What should I log for LLM08?
OWASP recommends detailed, immutable logs of retrieval activity so suspicious behavior can be detected and answered quickly. Record who asked, what was retrieved, and from which source.
Sources
Last reviewed: 18 October 2026.
