How do you write a tool evaluation report that others can trust?

State your method before your results, show the raw numbers, disclose conflicts and limits, and keep the decision separate from the data. A report others can trust lets a skeptical reader see what you did and reach their own view.

Outline

1. Purpose and decision. What choice does this report support, and who decides? One paragraph.

2. Conflicts. List any relationship between evaluators and candidate vendors, including free licenses. The FTC says to disclose financial or other relationships in a clear place, near the claim. Do the same here, at the top.

3. Method. Inputs (repositories, commits), how ground truth was set, tool versions and settings, number of runs, who labeled findings and how disagreements were settled. This is the section a skeptic reads first.

4. Criteria and weights. The scorecard as fixed before testing, with date. If weights changed, explain why.

5. Results. Per tool, per category: planted issues found, confirmed findings, false alarms in the labeled sample, run variation, setup effort and scan time. Give raw counts, not just percentages. Include precision and recall if you can compute them, with the definitions used.

6. What surprised us. Unexpected results, failures and workarounds. Honest surprises add credibility.

7. Limits. What the test did not cover: languages, scale, long term maintenance, support.

8. Decision and dissent. The recommendation, the reasons, and any disagreement in the team. Separate it from the data section.

Habits that build trust

• Name the version and date for every number.
• Keep the labeled sample as an appendix.
• Use the same units across tools.
• Avoid adjectives that outrun the data.
• Do not rank by a score if the criteria were weighted for one use case.

Habits that destroy trust

• Hiding a failed run.
• Changing criteria after seeing results.
• Reporting only the metric where the favorite wins.
• Quoting a vendor claim as if you measured it.

Share it wisely

If the report goes to a vendor, share facts about their product for correction, not conclusions for editing. Keep a copy of what changed. For outside readers, such as a board or an auditor, include the method appendix so they can judge it. Our post on inviting a third-party evaluation explains how to set expectations with outside reviewers.

How long should the report be

Short enough to be read, long enough to be checked. Two pages for the summary, decision and conflicts, and an appendix for the method, the labeled sample and the raw outputs. Executives read the first page. Skeptics read the appendix. Writing for both means neither has to ask you for more later.

Where Safeguard fits

Safeguard offers SBOM and dependency scanning, policy gates, and compliance and evidence automation, which can help you keep the records a report like this relies on. The free scan at gold.safeguard.sh/scan can be part of your test.

Quick check

Sources

• FTC: Disclosures 101 for social media influencers

Last reviewed: 19 October 2026.