Small models as security classifiers: a hands-on checklist

A small language model can act as a security classifier that screens prompts and responses for your policy categories at low cost and latency. It earns trust only after you test it on your own traffic, set thresholds, decide what happens when it fails, and monitor drift. It should sit beside other controls, not replace them.

What does this look like in practice?

Meta's Llama Guard paper is a public example. It describes an LLM-based safeguard built on Llama2-7b, instruction-tuned on a small, high-quality dataset, that classifies both user prompts and model responses against a safety risk taxonomy. It was evaluated on the OpenAI Moderation Evaluation dataset and ToxicChat, where the authors report it matched or exceeded existing moderation tools. Instruction tuning lets users adapt the taxonomy and output format, and the weights are released.

Notice what that gives you: a model that takes a taxonomy as input. The taxonomy is yours to write, and the quality of your results depends on it more than on the model name.

Why use a small model instead of a large one?

Cost, speed and control. A classifier runs on every request, so small size matters for latency and spend. It can also run inside your own boundary, which matters when the text being screened contains customer data. OWASP's page on sensitive information disclosure warns that instructions in a system prompt may not be honoured, which is a reason to put a separate checker in the path.

The checklist

1. Write the taxonomy first. List the categories you need to catch, such as secrets in output, personal data, policy violations, or attempts to override instructions, and define each with examples and non-examples.
2. Decide the position. Screen inputs, outputs or both. Llama Guard's design covers both.
3. Build an evaluation set from your own traffic. Label a sample by hand, including hard borderline cases, and keep it out of any tuning data.
4. Measure both errors. Missed detections create risk, and false alarms create friction and workarounds. Record the rate of each by category.
5. Choose thresholds per category. A secret in an output may justify blocking at a low score. A mild policy category may only warrant logging.
6. Pick the failure mode. If the classifier times out or errors, does the request pass or stop? Write it down per route.
7. Measure latency at your load, at the tail as well as the average.
8. Version everything. Record the model, taxonomy text, threshold and prompt template with each decision.
9. Log decisions with category, score and action, and sample them for human review weekly.
10. Test with adversarial variations in a staging environment: paraphrases, other languages, encodings and long inputs. Track what slips through and add it to the evaluation set.
11. Watch for drift. New product features and user behaviour change the traffic. Re-run the evaluation set monthly and after any model or taxonomy change.
12. Keep layered controls. Pair the classifier with deterministic checks such as pattern matching for secrets, permission checks in retrieval and limits on tool actions.
13. Treat the classifier as a supply chain component. Pin the source and file hash, and review its licence and updates like any dependency.

What can go wrong?

• Over-trusting benchmarks. Published results show performance on someone else's data.
• A taxonomy that is too broad. Vague categories produce inconsistent labels.
• No owner. Someone must read the sampled decisions.
• Classifier as sole defence. A single model is a single point of failure.

What to do

1. Write the taxonomy for one route this week.
2. Label 200 examples from real traffic and keep them frozen.
3. Run a candidate model and chart errors by category.
4. Set thresholds and failure modes, and test them in staging.
5. Ship in log-only mode first, then enforce category by category.

Quick check

Sources

• Llama Guard: LLM-based input-output safeguard for human-AI conversations (arXiv 2312.06674)
• OWASP GenAI, LLM02:2025 Sensitive Information Disclosure

Last reviewed: 18 October 2026.