A score anchor is a written description of what a 0, 1, 2, 3, or 4 means in one category. Write anchors in observable terms, such as counts and tasks completed, and test them with two reviewers before the trial. If reviewers disagree by more than one point, the anchor is unclear.
Why anchors matter
Without anchors, a 3 from one reviewer means "pretty good" and a 3 from another means "best I have seen so far." The totals look precise and are not. Anchors force a conversation before the scoring starts, when it is cheap, instead of after, when it becomes a political argument.
The rules for writing them
- Use things a reviewer can observe or count: tasks completed, steps required, items found.
- Make adjacent levels differ by one clear thing, not three.
- Describe 0 as a failure and 4 as strong but not perfect, so there is room above what you have seen.
- Avoid adjectives such as "good" or "excellent" without a measure beside them.
- Keep each anchor to a sentence or two.
Example: workflow fit
Measure a task your developers actually do: see a finding in a pull request, understand it, and fix it.
- Score 0: finding is not visible where developers work
- Score 1: visible, but needs a separate login and manual lookup to understand
- Score 2: visible in the pull request, with a link to a separate explanation
- Score 3: visible in the pull request with a clear explanation and a suggested fix
- Score 4: score 3, plus suppression with a reason, and no extra steps to rerun the check
Example: language and build coverage
Use your own list. Count how many of your top ten repositories the tool scans correctly on the first attempt.
- Score 0: fewer than 3 of 10
- Score 1: 3 or 4 of 10
- Score 2: 5 to 7 of 10
- Score 3: 8 or 9 of 10
- Score 4: all 10, with no custom configuration
Example: vendor support
During the trial, file three questions at different times, including one outside local business hours, and one that needs an engineer.
- Score 0: no useful answer within the trial
- Score 1: answered, but a day or more late or incomplete
- Score 2: answered in the stated time, needed follow up
- Score 3: answered correctly in the stated time
- Score 4: answered correctly, quickly, and the vendor volunteered a relevant next step
Test the anchors
Before the trial, have two people score the same sample: a short demo recording, or a result from a free scan such as the one at https://gold.safeguard.sh/scan. Compare. Any category where scores differ by two or more points needs a rewrite. Fix it, then lock the anchors.
Common problems you will find: anchors that mix two ideas (speed and accuracy), anchors with no numbers where numbers are available, and anchors that no vendor could ever reach, which squashes the scale.
Keep the evidence
Next to each score, record a note: what was observed, by whom, on which date. When a stakeholder challenges the result, the note is your answer. It also helps when you run the process again next year. For how the scores roll into a decision, see the weighted rubric example.
Quick check
Why use 0 to 4 and not 1 to 10?
Fewer levels are easier to define and easier to agree on. Ten levels invite false precision.
Who should write the anchors?
The people who will live with the tool, reviewed by someone who has not seen any vendor material, so the anchors do not favor a product.
Last reviewed: 19 October 2026.
