In one randomized trial, experienced developers using AI tools took 19 percent longer on real tasks, yet believed they had been about 20 percent faster. The result is narrow and dated, but it shows why feelings are a poor measure of productivity.
What the study did
METR assigned 246 real issues at random to allow or disallow AI use. The 16 participants were experienced developers working on large open source repositories they had contributed to for years, with tasks averaging about two hours. They recorded their screens and reported their own times. The main tool was Cursor Pro with Claude 3.5 and 3.7 Sonnet.
Before starting, the developers forecast a 24 percent speedup. After finishing, they estimated a 20 percent speedup. The measured result was 19 percent longer with AI.
Myth versus fact
Myth: this proves AI slows all developers down. Fact: it covers 16 people, familiar codebases and early 2025 tools. METR's own page says the results are out of date and points to a February 2026 update using newer tools. Read the update before quoting anything.
Myth: developers who feel faster must be faster. Fact: in this trial the feeling and the stopwatch disagreed.
Myth: a negative result means stop using the tools. Fact: it means measure before you decide. The study does not say what happens for newcomers to a codebase, for other task types or for newer tools.
Reasons the gap can exist
The study cannot tell you why for your team, and I will not guess at its causes beyond what it reports. In general, common sources of a perception gap include time spent prompting, reading and correcting output, and the satisfaction of seeing code appear quickly. Whether any of these applied here is a question for the full paper.
What to do with this
- Do not quote the 19 percent figure as general truth. Quote it as one trial with a specific setting.
- Collect your own timing data on your own tasks, for example change lead time from your pipeline.
- Pair speed with quality metrics such as change fail rate, since faster output that breaks more is not a win.
- Re-run the comparison when tools change. Results from a year ago may not hold.
DORA's guidance on its delivery metrics adds a related caution. Targets set on a metric invite gaming, so use measurements to learn, not to grade individuals.
A fair way to read it
Hold two ideas at once. The result is a real warning that self assessment can be wrong, and it is also a single study with limits its authors state. Both are true, and neither tells you what happens on your own code with today's tools. Collect your own data, and update your view as better evidence arrives.
Quick check
Does the METR study show AI always slows developers down?
No. It covers 16 experienced developers on familiar repositories with early 2025 tools, and the page says it is out of date.
What did developers believe about their own speed?
They predicted a 24 percent speedup and afterward estimated 20 percent, while measured times were 19 percent longer.
Sources
- METR, early 2025 AI and experienced open source developers study
- DORA, Software delivery performance metrics
Last reviewed: 19 October 2026.
