benchmarks
Safeguard articles tagged "benchmarks" — guides, analysis, and best practices for software supply chain and application security.
22 articles
Vulnerability Management SLA Benchmarks 2026
What credible 2026 vulnerability management SLAs look like across severity tiers, internet exposure, and reachability — with data from real programs.
Safeguard Griffin AI: Eval Benchmarks Published
Griffin AI's evaluation harness results published for the first time. Benchmark methodology, comparison against baselines, and what the numbers mean for production use.
Benchmark Reproducibility: Griffin AI vs Mythos
A benchmark you can't reproduce is marketing. A benchmark you can rerun on your own infrastructure is evidence. The reproducibility gap is wide.
Griffin AI vs Open Weights: The Eval Gap
Frontier models pass eval benchmarks that open-weight models miss by specific measurable margins. For security workflows, the gap matters.
Continuous Eval & Release Gating: Griffin AI vs Mythos
Evals that run once are marketing. Evals that run on every build are infrastructure. Griffin AI runs the harness on every change; Mythos does not describe one.
The Eval Culture Shift in AI Security
Two years ago, AI vendors shipped without evals. In 2026, the posture has shifted. Customers expect benchmarks. Vendors without them lose deals.
Golden Dataset Design: Griffin AI vs Mythos
Benchmark scores are only as honest as the dataset behind them. Griffin AI publishes golden-dataset design notes; Mythos-class tools rarely explain theirs.
Leakage Testing Methods For Security Benchmarks
A benchmark that the model has seen in training is a benchmark of memorisation. Specific leakage-testing methods separate generalisation from recall.
Regression Gates: Griffin AI vs Mythos
Every release risks making the model worse. Griffin AI's regression gates block bad builds before they ship. Mythos-class tools rarely describe a gate process at all.
Benchmark Contamination Concerns In Security Evals
When the test set is in the training set, the benchmark is broken. Security eval contamination is widespread and the mitigations are specific.
Refusal Rate Analysis: Griffin AI vs Mythos
A security AI that refuses too often is useless. One that refuses too rarely is dangerous. Griffin AI publishes calibrated refusal benchmarks; Mythos does not.
SEvenLLM Design And Coverage
SEvenLLM set out to measure how well LLMs handle Security Event analysis, the unglamorous day-to-day work of SOCs and IR teams. A design review of what the benchmark covers, how it was built, and where the coverage maps or does not map to real operations.