Skip to content

Is self-hosting an LLM cheaper than paying for an API?

Self-hosting beats a frontier API at about 16M tokens a day per H100, but beating cheap open-model APIs takes 240M+ a day and a busy GPU.

Self-hosting an open-weight model on one NVIDIA H100 is cheaper than a frontier API like Claude Sonnet 5.5 once you send it about 16 million tokens a day, but it takes roughly 240 to 400 million tokens a day to beat the cheap per-token APIs that serve the same open models. Nodus offers both sides, hourly GPUs and a per-token inference API (Beta) for open models such as gpt-oss-120b, so you can start per token and move to your own GPU when your volume crosses the line. What decides it is not the hourly price but how busy you keep the GPU.

You are comparing a fixed hourly bill against a variable per-token bill. A GPU you keep running costs the same whether it serves a million tokens or none. An API charges only for tokens, at a rate that already includes someone else’s idle hardware, redundancy and engineering.

To make the two comparable, we turn every API price into one blended price per million tokens. We use the reference mix the Nodus inference guide uses for its catalog: 3 input tokens for every output token. The blended price is (3 x input + output) / 4.

API and model Input per 1M Output per 1M Blended per 1M (3:1)
Anthropic Claude Sonnet 5.5 $2.00 $10.00 $4.00
OpenAI gpt-6.1-sol $2.00 $10.00 $4.00
Anthropic Claude Haiku 5.5 / OpenAI gpt-6-luna $0.10 $0.50 $0.20
Together AI, gpt-oss-120B $0.15 $0.60 $0.2625
DeepInfra, Llama 3.3 70B Instruct Turbo $0.10 $0.32 $0.155

Prices checked November 16, 2026, standard tier, from Anthropic’s pricing page (Haiku rate for prompts up to 100K tokens), OpenAI’s API pricing (short context), Together AI’s pricing and DeepInfra’s pricing. Batch tiers and cached input lower all of these.

Blended API price per million tokens at a 3 input to output mix: Claude Sonnet 5.5 and gpt-6.1-sol $4.00, gpt-oss-120B on Together AI $0.2625, Claude Haiku 5.5 and gpt-6-luna $0.20, and Llama 3.3 70B on DeepInfra $0.155. A large frontier model costs about 15 to 26 times more per token than an open model served by a per-token API.

That spread is the whole story. Against a $4.00 frontier price, almost any busy GPU wins. Against $0.155 to $0.2625, you are competing with providers whose job is packing many customers onto the same GPUs.

How many tokens a day does one GPU need to break even?

Section titled “How many tokens a day does one GPU need to break even?”

Divide the GPU’s daily cost by the blended price. We use one H100 because OpenAI’s gpt-oss-120b model card says the model fits on a single 80 GB GPU such as the H100. The H100 SXM is listed at $2.60/hr on the Nodus pricing page, checked November 16, 2026.

The arithmetic, once:

  • One H100 for a day: $2.60 x 24 = $62.40. For an average month of 730 hours: $2.60 x 730 = $1,898.00.
  • Against Sonnet 5.5 or gpt-6.1-sol: $62.40 / $4.00 = 15.6M tokens a day, or 474.5M a month.
  • Against Haiku 5.5 or gpt-6-luna: $62.40 / $0.20 = 312.0M tokens a day, or 9,490M a month.
  • Against gpt-oss-120B on Together AI: $62.40 / $0.2625 = 237.7M tokens a day, or 7,230.5M a month.
  • Against Llama 3.3 70B on DeepInfra: $62.40 / $0.155 = 402.6M tokens a day, or 12,245.2M a month.

Tokens per day one H100 at $2.60/hr must serve to match each API: 15.6M against Sonnet 5.5 or gpt-6.1-sol, 237.7M against gpt-oss-120B on Together AI, 312.0M against Haiku 5.5 or gpt-6-luna, and 402.6M against Llama 3.3 70B on DeepInfra. Replacing a frontier API pays off at small volume; undercutting an open-model API needs about 15 to 26 times more traffic.

Notice what the frontier row really compares. Switching from Sonnet 5.5 to a self-hosted gpt-oss-120b is a change of model, not just of hosting. It only saves money if the open model passes your evals. Our post on OpenAI fine-tuning vs open models runs the same kind of break-even for fine-tuned models, where the API side is pricier and the math favors your own GPU sooner.

Utilization decides it, because idle hours still bill. The daily totals above assume tokens arrive evenly around the clock. Real traffic does not. If your GPU is busy only 20% of the hours you pay for, it has to serve five times as fast while it is busy to hit the same daily total.

Here is the throughput your H100 must sustain while busy, in input plus output tokens per second, to tie each API. It is the break-even daily total spread over the busy seconds only.

Share of paid hours the GPU is busy vs Sonnet 5.5 ($4.00) vs gpt-oss-120B on Together ($0.2625) vs Haiku 5.5 ($0.20) vs Llama 3.3 70B on DeepInfra ($0.155)
100% 181 tokens/s 2,751 tokens/s 3,611 tokens/s 4,660 tokens/s
80% 226 tokens/s 3,439 tokens/s 4,514 tokens/s 5,824 tokens/s
50% 361 tokens/s 5,503 tokens/s 7,222 tokens/s 9,319 tokens/s
20% 903 tokens/s 13,757 tokens/s 18,056 tokens/s 23,298 tokens/s

Each cell is the daily break-even divided by 86,400 seconds, then divided by the busy share. For example, 237.7M / 86,400 = 2,751 tokens/s, and 2,751 / 0.2 = 13,757 tokens/s.

Throughput one H100 must sustain while busy to match each API: against Sonnet 5.5 it rises from 181 tokens/s at 100% busy to 903 at 20%, against gpt-oss-120B on Together AI from 2,751 to 13,757, and against Llama 3.3 70B on DeepInfra from 4,660 to 23,298. Halving how busy the GPU is doubles the throughput it needs, so a part-time GPU struggles to beat a per-token API.

We have not published a gpt-oss-120b throughput measurement of our own, so we will not tell you which cells an H100 can reach. That depends on your serving engine, prompt lengths, batch sizes and latency target. Measure it at your real traffic shape with an engine like vLLM (our explainer on what vLLM does covers why batching matters) and read your number against the table.

The same arithmetic gives an effective price for your own GPU. Suppose a load test shows 3,000 tokens per second while busy. At 30% busy, the cost is $2.60 / (3,000 x 3,600 x 0.3) x 1,000,000 = $0.80 per million tokens. At 90% busy it is $0.27.

The GPU and model are the same, yet the price per token triples at the lower utilization. Only the 90% case gets close to the $0.155 to $0.2625 that open-model APIs charge.

What does self-hosting cost besides the GPU?

Section titled “What does self-hosting cost besides the GPU?”

More than the GPU line suggests, and none of it shows up in the break-even above. We leave these unpriced because they depend on your team, but count each one before you decide:

  • Redundancy. One GPU is one point of failure. Production serving usually means a second replica, which doubles the GPU bill unless traffic justifies both.
  • Headroom for peaks. Sizing for your busiest hour lowers average utilization, which pushes you down the table above.
  • Engineering time. Choosing an engine and quantization, tuning batch sizes, upgrading models and keeping latency in bounds is ongoing work.
  • Startup and storage. Every new replica has to load tens of gigabytes of weights before it serves, and the weights and logs need storage.
  • Monitoring, security patching and on-call for a service you now own.

A per-token API folds all of this into its price. That is why its rate can look high per token and still be the cheaper option for most teams.

Self-host when the volume is steady and high, or when something other than price requires it. We would pick a dedicated GPU in these cases:

  • You are replacing a frontier API with an open model that passes your evals, and you send more than about 16M tokens a day per GPU you would run.
  • Your traffic is steady enough to keep a GPU busy most hours, for example batch classification, embeddings or overnight document processing that you can queue.
  • You serve a fine-tuned model or several LoRA adapters that no per-token API hosts.
  • Data rules, latency or guaranteed capacity matter more than the per-token rate.

Stay on a per-token API when traffic is spiky, below the break-even, or when an open-model API already serves the model you want. If you are weighing buying a workstation instead of paying by the hour, our post on running an LLM locally vs in the cloud does that hardware math.

Start per token, then move the same model onto a GPU when your logs show the volume. The Nodus inference API is in Beta and is OpenAI-compatible. Its rates per model, including cached input, are shown in the console’s Models catalog and by GET /v1/models; we do not repeat them here because they update as the catalog changes. These commands are copied from the inference guide:

Terminal window
$ nodus inference models
$ nodus inference chat --model nodus/gpt-oss-20b --max-tokens 64 "What is the capital of France?"

chat prints the token counts and the charge for the request, so a day of real traffic gives you the input to output mix and the daily volume this post’s math needs. A named endpoint adds a spending cap: nodus create inferenceendpoint support-bot --model nodus/gpt-oss-120b --max-cost 50 answers the next request with 402 BudgetExceeded once its requests have spent $50. When your daily tokens clear the break-even and your load test clears the busy-throughput row you need, run the model on an hourly GPU from the pricing page instead. For dedicated inference capacity, the guide says Nodus sets up a custom plan.

For light personal use, yes, if the model already fits on hardware you own, since the extra cost is mostly electricity. For a product’s traffic, a local or self-hosted model is only cheaper above the break-even volume, and a cheap open-model API usually wins below a few hundred million tokens a day per GPU.

Does output-heavy traffic change the break-even?

Section titled “Does output-heavy traffic change the break-even?”

Yes. Output tokens cost 3.2 to 5 times more than input on every API above, so a chat or agent workload with long answers has a higher blended price and a lower break-even. Recompute the blend with your own measured mix instead of 3:1.

They move it toward the API. OpenAI and Anthropic both list batch processing at half the standard price on their pricing pages, which doubles the tokens a day your GPU must serve to break even on work that can wait.

Multiply the GPU cost by the number of GPUs the model needs. A model that needs 8 GPUs needs 8 times the daily tokens to break even, which is why very large open models are usually cheapest through a per-token API, such as the Nodus inference API for models in its catalog, unless traffic is enormous.

More from the blog