AI Security
Fine-Tune Drift Measured On Eval Sets
Fine-tuning to improve one task frequently regresses others. Without eval harnesses, the regressions ship. The measurable drift is larger than vendors admit.
Mar 20, 20262 min read
Deep dives, practical guides, and incident analyses from engineers who build Safeguard. No fluff, no vendor FUD — just what you need to ship secure software.
Fine-tuning to improve one task frequently regresses others. Without eval harnesses, the regressions ship. The measurable drift is larger than vendors admit.
The difference between grounded reasoning and hallucinated reasoning is not eloquence — it's citation. A look at how Griffin AI anchors every claim.
Two dozen AI guardrail vendors in 2023. A much smaller set in 2026. The consolidation has pattern — integrated platforms beat standalone guardrails.
An auto-fix that closes a vulnerability and breaks the build is not a fix. Breaking-change awareness separates auto-PRs that ship from auto-PRs that get reverted.
LLM-suggested package names that do not exist are a registered attack vector in 2026. Here is where hallucination rates sit today and how to contain them.
Prompt injection is the defining AI security problem of this generation. The defences are structural, not cosmetic — and the architectural choices show.
AI-for-security metrics that show up on board slides are different from the ones engineers use day-to-day. Designing both sets properly is the work.
Synthetic eval benchmarks are controllable. Real-world data is messy. The gap between performance on each is usually large, and vendors prefer one over the other for a reason.
Crypto misuse is not about broken algorithms. It is about misused parameters, missing checks, and the gap between "it compiles" and "it is secure."
Weekly insights on software supply chain security, delivered to your inbox.