You added AI attribution to your pipeline. The dashboard says 8.1% of added lines came from AI-co-authored commits. Someone in a governance meeting will now round that to "8% of our code is AI-written" and put it in a policy document.
Those are not the same claim, and the distance between them matters more than the number does.
What the metric actually counts
Commit-trailer attribution works at commit granularity. A commit either carries an assistant trailer or it does not. When it does, every line that commit added is counted.
That gives you a precise, reproducible sentence: lines added by commits an assistant co-authored. Everything people want to do with the number — capacity planning, review policy, audit response — depends on whether that sentence is close enough to the one they have in their head.
Usually it is not, in four specific ways.
1. A later human rewrite does not subtract
An assistant writes a 400-line module. Over the next month three engineers refactor it until 60 of those lines survive. The attribution total still says 400.
There is no mechanism by which it could say otherwise. Git tracks that lines changed, not that they replaced AI-written lines specifically. git blame would tell you who last touched each line, but "last touched" is a different question again — a formatting pass rewrites every line in a file without changing a single decision in it.
This biases the number up relative to "how much AI-authored code is in the tree today".
2. Stripped trailers erase whole commits
Any convention that rewrites commit messages — a linter enforcing a strict format, a merge bot generating its own message, a wrapper that tidies output — removes the only evidence that a commit was AI-assisted. Those commits count as fully human.
This biases the number down, and unlike the first effect it is invisible. A rewritten total looks exactly like a low total.
3. Multi-assistant commits count differently in different places
A commit co-authored by two assistants contributes its additions once to the overall attributed total, and once to each model's row in the breakdown. Sum the rows and you exceed the total.
That looks like an arithmetic bug and is not one. "How much did this specific model touch" and "how much of this is AI-touched at all" have different denominators on purpose. Normalising them into agreement would mean picking a rule — first assistant wins, split the lines evenly — and every such rule invents information the commit does not contain.
4. Vendored and generated code inflates additions
A commit that checks in a lockfile update, a regenerated protobuf, or a vendored dependency adds thousands of lines. If the assistant helped write the one-line config change that triggered the regeneration, the whole diff is attributed.
Excluding generated paths from the denominator is possible and usually worth doing, but every exclusion list is a judgement call that has to be recorded, or the metric stops being reproducible.
Label it in the data, not in the footnote
The mitigation is boring and it works: store the caveat next to the number, in the same record, where it travels with the data into whatever reads it next.
{
"ai_attributed_additions": 14435,
"ai_additions_scanned": 178374,
"ai_attributed_share": 0.081,
"ai_attribution_basis": "commit trailers; per-commit, not per-line; a floor rather than an estimate"
}
A footnote in the dashboard UI protects only the person looking at the dashboard. A field in the record protects the person who exports it to a spreadsheet, the API consumer, and the auditor who receives a JSON dump six months from now with no UI at all.
Name the fields to match. attributed_additions reads as a term of art and invites the question "attributed how?". ai_written_lines reads as a fact and invites no questions whatsoever, which is precisely the problem.
What the number is genuinely good for
Given all of that, is it worth collecting? Yes — for the questions it can answer honestly.
Trend over time. The biases above are roughly stable within one repository under one set of conventions, so the direction of change is meaningful even when the absolute value is not.
Model inventory. "Which assistants have contributed to this codebase, and when did each first and last appear" is answered exactly, with no estimation involved. For an organisation that needs to know whether a model with a since-revised licence ever touched a shipping product, that is the whole question.
Review targeting. Commits with assistant trailers are a defensible population to sample for a review policy. Not because AI-written code is worse, but because "we review a fixed percentage of assisted commits" is a control you can state, implement and evidence — which is more than most AI code policies manage.
Establishing a floor. "At least 8% of added lines came from assisted commits" survives cross-examination. "8% of our code is AI-written" does not.
The test that keeps it honest
Write the assertion into your test suite rather than your documentation:
- a repository with no git history produces no attribution record, rather than a record asserting 0%
- a repository that was scanned and found nothing produces a record with
commits_scanned > 0andattributed_commits == 0
Those are different findings — "we could not look" and "we looked and found none" — and a system that renders them identically will eventually be asked to defend a zero it never actually measured.
