Before you chunk and embed anything, scan it. Indexing copies content into a new store and makes it searchable, so a leaked key or a hidden instruction in a document becomes easier to find and easier to abuse. Run the checks below as the first stage of your pipeline.
Why this stage matters
Once text is embedded, it is hard to inspect. You cannot read a vector. OWASP's guidance on vector and embedding weaknesses recommends validating knowledge sources, auditing the knowledge base for hidden content and poisoning, and accepting data only from trusted sources. Its examples include a résumé with hidden white on white text that instructs a screening system to recommend the candidate, with the mitigation of extracting text in ways that expose hidden content.
Its sensitive information guidance recommends scrubbing or masking sensitive content before it enters training, and applying least privilege to what the model can reach.
The checklist
1. Secrets. Run a secret scanner over every file and its history if you index history. Block files that match, rotate anything found. Include environment files, notebooks and config dumps.
2. Personal data. Customer exports, support transcripts and test fixtures with real records are common. Detect obvious patterns and exclude or mask them.
3. Source trust. List where documents come from. Internal repository, shared drive, uploaded file, web page. Treat anything a user can edit as untrusted. A wiki anyone can edit is a way to write into your index.
4. Hidden text. Extract text the way a reader would see it, and also the raw text. Compare. Flag white on white text, zero size fonts, comments, and metadata fields. Strip or reject.
5. Instruction like content. Search for phrases aimed at a model, such as "ignore previous instructions." Flag for review. This will not catch everything, but it catches lazy attacks.
6. File types. Decide what you accept. Binary blobs, archives and executables rarely belong. Parsing untrusted files is itself a risk, see our note on document parsing advisories.
7. Size and duplication. Huge or repeated files distort rankings. Cap and deduplicate.
8. Licence and ownership. Check that you may use the content for this purpose, especially third party documents.
9. Classification. Tag each document with a sensitivity level so retrieval can filter. See permission aware retrieval.
Make it a gate
Failures should stop the document, not log and continue. Keep a quarantine folder and a report. Humans review the quarantine weekly.
Re-scan on update
Run the same checks on every changed file, not only on first load. Poisoning can arrive in an edit.
The tools around the index
Your pipeline has its own dependencies: parsers, embedding libraries, a vector extension, a model runner. Those are software too. Safeguard's SBOM and dependency scanning can inventory them, and the free scan at gold.safeguard.sh/scan is a quick way to see what is in a project.
Quick check
Is a secret scanner enough?
No. It covers one item in this list. Hidden text, personal data and untrusted sources need their own checks.
Do I need this for a local only index?
Yes. Local does not mean trusted. Anything that lands in the index can influence answers.
Sources
Last reviewed: 19 October 2026.
