Before the pilot ends, export your results, record the exact configuration you tested, list every vendor promise made during the trial, and turn them into contract terms. Production behaviour often differs from a pilot, so the goal is to make the signed product match the one you evaluated.
Why the gap appears
During a pilot the vendor gives close attention. Settings are tuned, support answers fast, and a sales engineer may fix things by hand. After signing, you get the standard product and standard support. Whatever made the pilot succeed that is not written down can disappear.
What to capture
| Item | How to capture it |
|---|---|
| Results | Export findings, noise rates and time to triage in a standard format |
| Configuration | Policies, rules, exclusions, severity mappings, integrations |
| Environment | Languages, repos, CI platforms and versions tested |
| Promises | Every commitment from calls and emails, with date and speaker |
| Support | Response times observed, tickets raised and how they ended |
| Gaps | Features that were missing, with the vendor's stated plan |
Store this in one folder with a one page summary.
Steps
- Freeze the baseline. Export the pilot configuration and keep a copy. Ask the vendor to confirm it is reproducible in the production tenant.
- Compare to your success criteria. Score the pilot against the criteria you wrote before it started. See pilot success criteria before the trial.
- List promises. Mark each as already tested, covered by documentation, or a new commitment.
- Convert commitments to terms. Anything that influenced your decision and is not in public documentation belongs in the order form or security exhibit.
- Plan the move. Decide whether the pilot tenant becomes production, or a clean tenant is created. If clean, check that the configuration carries across.
- Agree onboarding. Name the rollout order, the owner on each side and the first review date.
Clauses worth asking for
- A support commitment equal to what you experienced, or a description of the difference.
- Data deletion for the pilot tenant if you do not sign. Pilot data is real data.
- Exit and export terms, since they are easiest to negotiate before signing. See appsec contract exit plan.
- Notice of feature changes that affect your configuration.
Measure noise the same way after launch
Use the same metrics you used in the trial, such as the share of findings that were actionable. Track them again at 30 and 90 days to see whether production matches the pilot.
Try it yourself
The reachability demo shows how filtering to reachable findings reduces the list. When you export pilot results, note which findings were reachable, since that count is a useful baseline for later comparison.
Click through it at your own pace, or open it full screen.
Safeguard offers reachability analysis and a free scan at gold.safeguard.sh/scan.
Quick check
What happens to pilot data if we do not buy?
It should be deleted on a stated timeline, with written confirmation. Agree that before the pilot starts.
Should verbal promises go in the contract?
If they influenced your decision, yes. Otherwise rely on public documentation only.
Last reviewed: 19 October 2026.
