The Software Assurance Reference Dataset (SARD) is a NIST SAMATE collection of test programs with documented weaknesses. Buyers can run candidate tools against selected cases, compare reported findings with the documented ones, and learn where each tool is strong or blind.
What it contains
NIST describes SARD as a growing collection of test programs with documented weaknesses. Its page reports more than 450,000 test cases, ranging from small synthetic programs to large applications, written in C, C++, Java, PHP and C#, and covering more than 150 Common Weakness Enumeration classes.
NIST says it is used to help tool developers improve their tools and to help end users find suitable tools for their projects. The project also accepts submissions of vulnerable software artifacts.
Why documented weaknesses matter
A test case with a documented weakness gives you ground truth. You know a particular flaw is present at a particular place. A tool either reports it or does not. That is the basis for counting missed issues, which is hard to do on real code where nobody knows what is hiding.
Using it in a bake-off
- Pick the languages you really use. If you write mostly one language, ignore the rest.
- Pick the weakness classes that matter to you. The CWE list is maintained by MITRE with sponsorship from CISA, and its categories give you a shared vocabulary for tool coverage.
- Select a sample of cases, and record which ones you chose and why.
- Run each tool on the same cases with defined settings.
- Compare reported findings with the documented weaknesses and note both misses and unexpected results.
Keep the sample small enough that a person can look at every unexpected result. Unexpected does not always mean wrong, because a synthetic case may contain a genuine flaw nobody documented.
Limits you should state
Synthetic programs are not your codebase. Frameworks, build tools and coding patterns in real applications differ, and tools that do well on small cases can behave differently at scale. Treat results as one input, and pair them with a run on your own repositories. Our post on synthetic benchmarks versus your own code explains how to weigh both.
Also remember that the page does not describe how any particular case was labeled, so read the documentation for each case you use.
Questions to ask when a result cites SARD cases
If a vendor or reviewer says a tool was tested on these cases, ask which cases, how many, and for which weakness classes. Ask whether the tool was tuned for them, and whether the same configuration was used for every tool. Ask how many unexpected findings were examined by hand. A claim that says "tested on a large public dataset" with no sample description tells you almost nothing, because the dataset spans many languages and weakness types and a flattering subset is easy to pick.
Where Safeguard fits
Safeguard offers SBOM and dependency scanning, reachability analysis and zero-day discovery. A free scan at gold.safeguard.sh/scan lets you try it on your own code next to a test suite run.
Quick check
Does a good result on SARD cases prove a tool is good?
No. It shows how the tool handles those cases. Real code may behave differently, so test on your own repositories too.
Sources
Last reviewed: 19 October 2026.
