A fair comparison lists every cost line on both sides. Hosted AI bills by meters such as input and output tokens. Self-hosting replaces those with hardware, utilization, engineering time and security work. Compare cost per completed task at your real volume, not a rate per token against a GPU hourly figure.
Hosted: the meters
A hosted API charges by use. The pricing page of one provider shows how many meters can apply: input, output, cache reads and writes, batch, region, speed tier and tool use. Limits matter too, since your tier decides how much you can send per minute. See how to read an AI pricing page for the full list.
What you do not carry on the hosted side is the hardware, the serving stack and the on-call burden for the model itself. What you do carry is dependence on the provider's limits, terms and model lifecycle.
Self-hosted: the lines
| Cost line | Notes |
|---|---|
| Accelerators | Bought or rented, whether busy or idle |
| Utilization | An idle GPU still costs; low traffic raises cost per task |
| Memory fit | Weights and the key value cache must fit, which is why quantization comes up |
| Serving software | Needs patching and upgrades |
| Engineers | Deployment, monitoring, capacity planning, incident response |
| Security work | Image scanning, network isolation, access control, model provenance |
| Evaluation | You own quality testing for every model change |
| Redundancy | A second copy for failure, which doubles idle capacity |
None of these needs a number to be on your worksheet. Fill in your own quotes.
The utilization question
The most common error is comparing a hosted rate to hardware cost at full use. Real traffic has peaks and troughs. If your demand is steady and high, owned capacity can be used well. If it is spiky or small, a meter that stops when you stop may fit better. This is general reasoning, so test it with your traffic shape.
Questions that decide it
- How many completed tasks per month, and how variable is that?
- Do you have data rules that prevent sending content to a provider? If so, the comparison may be moot.
- Do you need a specific model that is only offered one way?
- Who is on call when the serving stack fails at night?
- How quickly can you move if the provider changes terms or retires a model?
- What does a month of limits or an outage cost your business?
Security in either case
Self-hosting moves model security work onto you. Serving ports need locking down, images need scanning and weights need provenance checks. Our guides on locking down an inference port and scanning a GPU inference container cover the basics. Hosted use moves risk to key management, data handling terms and spend controls, covered in spend cap as a security control.
Build the worksheet
Put both options side by side with one row per line above, a column for owner and a column for source. Use measured tokens per task from a pilot. Date it. Include a row for what you would do to exit each option. For a broader frame, see build vs buy worksheet.
Safeguard's SBOM and dependency scanning can help inventory what your serving stack includes, which feeds the security lines of the self-hosted column.
Worked example: filling the worksheet with placeholders
Use letters, not figures. A team compares a hosted API with running one model on rented GPUs.
| Line | Hosted | Self-hosted |
|---|---|---|
| Usage meters | T tokens x rates, plus cache and batch effects | None |
| Hardware | None | G GPUs x hours x rate, busy or idle |
| Utilization | Not your concern | U percent, which scales cost per task |
| Engineering | Integration only | E hours per month for serving, upgrades, on-call |
| Security work | Key management, data terms | Image scanning, isolation, provenance, patching |
| Redundancy | Provider's | Second copy, R |
| Evaluation | When you change model | Every change |
| Exit effort | Switch provider | Move hardware or contracts |
Cost per task is then (all monthly lines) divided by (tasks completed per month). Run it for three volume cases. The crossover often moves with utilization, so show U explicitly and try a low, middle and high value.
Decision rules you can write down
- If data rules forbid sending content out, self-hosting or a private deployment is the only path, and the comparison is about which one.
- If traffic is small or spiky, favor the option whose cost follows usage.
- If you need a model that is offered only one way, that settles it.
- If you lack staff to run an on-call rotation for the serving stack, price that gap honestly.
A pilot before a decision
Run the hosted option for a month with real tasks and record tokens per task. Stand up a small self-hosted test with the same evaluation set and record throughput and memory. Compare on the same tasks, not on vendor claims. Write down what would change your mind, such as a volume threshold.
Presenting the result
Show leadership one chart with cost per task for each option at three volume cases, and a short list of non-cost factors: data rules, staffing, model availability and exit effort. Put your assumptions on the same page, with sources and a date. A comparison that hides assumptions invites a debate about the numbers, while one that shows them invites a debate about the assumptions, which is the useful one.
Exit and lock-in
Whichever route you choose, write down how you would leave it. For a hosted API, that means keeping prompts, evaluation sets and output formats portable, so another model can be tested quickly. For self-hosting, it means keeping the serving configuration in code and avoiding hardware specific tricks you cannot reproduce elsewhere. The cost of leaving is part of the cost of staying, and a team that has rehearsed an exit negotiates and plans from a stronger position.
Review the decision on a schedule, perhaps every six months, because models, prices, limits and your own volumes all move.
What this means for you
If you are weighing self-hosting against a hosted API, the honest answer is usually that it depends on your volume, your constraints and your staff, and that you will not know until you measure. Treat vendor claims and blog posts, including this one, as a list of questions rather than a verdict.
What helps most is to turn both options into the same unit, which is cost per completed task, and to be explicit about the assumptions behind each figure. A team that does this well can explain its choice to finance, security and engineering in one page.
Common mistakes
A frequent mistake is to compare a hosted rate per token with the price of a GPU at full load, which assumes the hardware is always busy. Another is to leave engineering time and on-call burden off the self-hosted side, because they do not appear on an invoice. A third is to forget the security work of running a serving stack: patching, isolating ports, scanning images and checking where weights came from. A fourth is to ignore how a model change affects either option, since a new model may need different hardware or a new evaluation. A fifth is to decide once and never revisit.
A step by step approach
- Define the workload: tasks per month, prompt and output sizes, and peak demand.
- Price the hosted route using the provider's current page, listing every meter that applies.
- Price the self-hosted route with real quotes, including idle time and a redundant copy.
- Add people costs for both routes, even if one is small.
- List the constraints that might decide the matter regardless of cost: data rules, model availability and latency needs.
- Run a small pilot of each and compare on the same evaluation set.
- Write the decision with its assumptions, and set a review date.
The result may be a mix, with sensitive or steady work in one place and spiky work in another.
Try it yourself
Click through it at your own pace, or open it full screen. Self-hosting means owning a stack of packages. The demo shows an inventory of dependencies, which is the first view you need to estimate the security upkeep on the self-hosted side.
Quick check
Is self-hosting always cheaper at scale?
Not necessarily. It depends on utilization, staffing and security work. Build the worksheet with your own numbers.
What is the most overlooked self-hosting cost?
Idle capacity and the engineering and security time to run the stack. Both are easy to leave off a comparison.
Can I compare per-token rates directly with GPU cost?
Not safely. Convert both to cost per completed task at your expected volume.
Sources
Last reviewed: 19 October 2026.
