Cost per token tells you the price list. Cost per completed task tells you what your product costs to run. Define a task in business terms, attribute every token call to it, include retries and failures, and divide total spend by successful tasks. That number is the one to budget and optimize.
Why per token comparisons mislead
Two models at the same price per token can differ widely in cost per task. One may need three tool calls where another needs eight. One may reason for pages before answering. One may fail and trigger a retry. The token counting documentation also notes that newer model generations tokenize the same text into roughly 30 percent more tokens in one comparison, so an unchanged price per token can still raise the bill.
Define the task
A task is the unit a user or business cares about: a resolved support ticket, a reviewed pull request, a summarized contract, a generated report. Write the definition down, along with what counts as success. Without a success definition you cannot divide.
Attribute every call
Give each task a task ID and pass it with every model call that contributes to it, including planning calls, tool loops, retries, and judge calls. Store it with the usage fields from each response: input, cached input, cache writes, output, and thinking tokens.
For org level reconciliation, the Usage and Cost Admin API in the provider docs groups usage by model, workspace, and API key, so you can check that your per task totals roughly match the invoice.
The formula
Cost per task is the sum of token costs across every call for the task, plus any tool or search charges, divided by the number of tasks that succeeded. Failed tasks stay in the numerator and stay out of the denominator. That is what makes a flaky pipeline look as expensive as it really is.
Track the median and a high percentile. A mean can hide a long tail of runaway agent loops.
What moves the number
- Prompt structure and caching, which cuts repeated input cost.
- Effort and reasoning level, which changes output volume.
- Model choice per step.
- Retry rate and loop length.
- Batch processing for work that can wait.
Change one at a time and re-measure against the same evaluation set.
Set budgets at the task level
A per task ceiling gives you a clean guardrail: stop the loop and return a partial result when a task passes its cap. Spend limits at the workspace level are a coarser backstop, not a substitute.
Report it to finance
Cost per task, by feature, by month, is a number a finance partner can use for pricing and margin discussions. A monthly token total is not.
Quick check
What if tasks vary hugely in size?
Segment them by size or type and report each segment. A single average across very different tasks is not actionable.
Should failed tasks be excluded from cost per task?
Keep their cost in the total and exclude them from the count of successes. Otherwise the metric flatters the system.
Sources
Last reviewed: 19 October 2026.
