At what volume does self-hosting start winning?

There is no universal number because it depends on utilisation rather than volume. The honest test is whether a dedicated instance would run at high utilisation around the clock — if it would sit idle at night, the arithmetic rarely works.

Are open-weights models good enough?

For most production tasks, yes. Classification, extraction, summarisation and routing are handled well. The gap is narrowest on narrow tasks and widest on hard reasoning.

Comparison

API vs self-hosting an open model

APIs bill per token; self-hosting bills per GPU-hour whether you use it or not. Where the crossover actually is, and the costs nobody puts in the spreadsheet.

The short answer

Use an API unless you have high, steady utilisation or a hard data-residency requirement. APIs bill per token, so idle time is free; self-hosting bills per GPU-hour whether or not a request arrives. Bursty or low-volume workloads are almost always cheaper on an API, and the engineering cost of running inference reliably is the line most comparisons omit.

At a glance

Hosted API compared with Self-hosting
	Hosted API	Self-hosting
Cost model	Per token used	Per GPU-hour, used or not
Idle cost	Zero	Full
Time to first request	Minutes	Days to weeks
Scaling	Automatic, within rate limits	Your problem
Data leaves your infrastructure	Yes	No
Model choice	What the provider offers	Any open-weights model
Model deprecation	On the provider’s timeline	On yours
Ongoing engineering	Minimal	Real and continuous

When to choose which

Choose Hosted API when

Traffic is bursty or uncertainWhich describes almost every product before it is established. Paying only for what you use is worth a great deal when you cannot predict usage.
You want the most capable modelsThe frontier is closed-weights. Open models are excellent and generally trail the best hosted models on the hardest tasks.
The team is smallRunning inference reliably is a specialism. Every hour spent on it is an hour not spent on the product, and that trade is rarely worth it early.

Choose Self-hosting when

Utilisation is high and steadyThis is the real condition. A GPU running near capacity around the clock can beat API pricing substantially; the same GPU at ten percent utilisation is a large monthly bill for nothing.
Data cannot leave your infrastructureA contractual or regulatory constraint, not a preference. Where it genuinely applies, it settles the question regardless of cost.
You need a model that will not be deprecatedWeights you hold do not disappear on somebody else’s schedule. For a product with a long support commitment this is a real consideration.
The task is narrow and a small model does itA fine-tuned small model on dedicated hardware, running one well-defined task at volume, is where self-hosting is most clearly right.

What it costs either way

The comparison people make is per-token cost against per-token cost, which is the wrong comparison. An API bills only for tokens; a GPU bills for time.

Model it as utilisation. Take your monthly token volume, work out the GPU-hours needed to serve it, and compare against the hourly rate of the instance — including the hours where no request arrives.

Add the parts that never make the spreadsheet: engineering time to build and maintain the serving stack, on-call, redundancy for availability, and the cost of a model upgrade that on an API is a config change.

A useful sanity check: if the API bill is under a few thousand dollars a month, self-hosting is very unlikely to be cheaper once engineering time is counted honestly.

The mistake people make

Comparing at peak utilisation

Self-hosting looks compelling when you model a GPU running flat out, because that is its best case. Real traffic has nights, weekends and quiet periods, and the instance bills through all of them. Model your actual utilisation curve rather than your peak, and the crossover moves a long way to the right.

How to decide

1Measure your real monthly token volume, not your projected one.
2Work out the utilisation a dedicated instance would run at, using your actual traffic shape including quiet hours.
3Add engineering time at a real rate — building and running inference is not free, and it is continuous rather than one-off.
4If it is close, stay on the API. The optionality of switching models without redeploying is worth something, and it is not in either column.

Price it yourself

Frequently asked questions

At what volume does self-hosting start winning?: There is no universal number because it depends on utilisation rather than volume. The honest test is whether a dedicated instance would run at high utilisation around the clock — if it would sit idle at night, the arithmetic rarely works.
Are open-weights models good enough?: For most production tasks, yes. Classification, extraction, summarisation and routing are handled well. The gap is narrowest on narrow tasks and widest on hard reasoning.