Using LLMs for security work comes down to a few habits: give context, require evidence, label untrusted data, validate output, keep privileges small, and verify in ways that do not depend on the model. These ten short answers cover the questions that come up most.
What should every security prompt contain?
A role, a narrow scope, the trust boundaries, the code or data labeled as data, an evidence rule, and an output format. See a prompt template for security code review.
How do I stop it inventing findings?
Allow "no findings" and "unsure", require verbatim quotes, run a retraction pass, and compare runs. Anthropic notes these reduce hallucinations and do not eliminate them. See cutting false positives.
Can I paste production logs and code?
Only if your policy and the tool's data terms allow it, and after removing credentials and personal data. See what not to paste.
Do XML tags protect against prompt injection?
No. They help structure. OWASP says prompt labels alone do not enforce the boundary between trusted and untrusted content.
Is prompt injection solvable?
The UK NCSC argues it may never be fully mitigated and should be managed by design. OWASP says no fully reliable prevention method is known. Plan for impact reduction.
How do I limit the damage?
Least privilege, no open ended tools, scoped credentials, human approval for high impact actions, and authorization enforced in downstream systems. These are the OWASP excessive agency recommendations.
Should model output go straight into tickets or scripts?
No. Validate it as you would user input, and encode it for its destination. See structured JSON output for triage.
Can an LLM do threat modeling?
It can brainstorm and draft. A team has to validate the system model and own the decisions. See can an LLM replace a threat modeling session.
How do I know my prompt is good?
Test it on a small answer key built from your own past findings, including clean cases, and keep the score with the prompt version. See write the rubric first.
What should not be left to an LLM at all?
Facts that tools can compute exactly. Whether a vulnerable function is reachable from your code, for instance, is a program analysis question. Safeguard's reachability analysis and dependency scanning handle that kind of question, and a free scan is available at gold.safeguard.sh/scan.
A starting routine for a small team
If you want one thing to adopt this week, use this routine. Save one review prompt template in the repository. Require a quote for every finding. Keep a spreadsheet of confirmed, rejected, and unclear results for a month. Review the spreadsheet at the end and adjust the scope line of the prompt based on where the misses and inventions concentrate. This costs little, makes the weak spots visible, and gives you real numbers to bring to anyone asking whether the tool is worth the risk.
Quick check
What is the single most useful habit?
Require a quote for every claim and check it. It makes wrong answers visible quickly.
Sources
- Reduce hallucinations (Anthropic)
- LLM Prompt Injection Prevention Cheat Sheet (OWASP)
- Prompt injection is not SQL injection (UK NCSC)
- LLM06:2025 Excessive Agency (OWASP GenAI)
Last reviewed: 19 October 2026.
