A 30 day proof of concept needs four weeks with a different job each: set up and success criteria, accuracy testing, developer pilot, and decision. Write the success criteria before the vendor touches your code, limit the scope, and decide the go or no-go rule in advance.
Before day one
Do these first:
- Pick two or three finalists. More than that spreads your team thin.
- Write success criteria with numbers (see success criteria to set before the trial).
- Choose the test repositories and the pilot team.
- Agree a shared evaluation environment so every vendor gets the same conditions.
- Name one person who owns the process and one who can say yes to the purchase.
Week 1: set up and baseline
Install each tool on the same set of repositories. Record effort: how long did it take, how many support requests, how many configuration changes were needed to get a first result. These numbers are part of your score.
In the same week, create your baseline. Take a handful of repositories and have your best engineer list the real issues by hand. You will compare tool output against this list.
Week 2: accuracy and coverage
Run each tool against the baseline repositories and against a set of repositories with known planted flaws. Count what was found, what was missed, and what was noise. Details are in the post on measuring false positives in a trial.
Also test coverage: do the tools handle your languages, monorepos, private registries, and unusual build setups?
Week 3: developer pilot
Put the tool in front of a real team on real work. Ask them to use it in pull requests for two weeks, not just click through a demo. Collect:
- How many findings they acted on
- How many they suppressed, and why
- Time added to the pull request process
- A short survey on usability, the same questions for every tool
The developer experience test describes the survey.
Week 4: integration, commercial terms, decision
Test reporting, single sign-on, API, and the alerting or ticket flow you need. Request a final pricing proposal based on your numbers. Check two customer references.
Hold the decision meeting at the end of the week. Score using the rubric you fixed in week 0. Compare the total against your go or no-go threshold, then discuss concerns that do not fit the scores.
Rules that save the trial
- Do not extend. If a vendor needs more time, that is evidence about setup effort.
- Do not accept a vendor run demo on vendor chosen code as part of the score.
- Keep a log of every issue raised and how fast it was fixed.
- If nobody on your side has time to run the trial, postpone it. A rushed proof of concept costs more than a delay.
Safeguard's free scan at https://gold.safeguard.sh/scan can be run on a public repository in week 1 to see the output format.
Quick check
Is 30 days enough?
For one tool category, usually yes. If you cannot get through the developer pilot, shorten the scope, not the pilot.
Should we test production repositories?
Choose a mix: some representative production repositories, with approval, and some test repositories with known flaws. Check each vendor's data handling terms first.
Last reviewed: 19 October 2026.
