Skip to content

How much does LLM inference cost per million tokens?

1M tokens costs $0.09 to $8 of input and $0.18 to $40 of output by tier, but resent context and caching decide most real bills.

One million tokens costs about $0.09 to $0.15 of input and $0.18 to $0.60 of output on small and open-model APIs, and $2 to $8 of input and $10 to $40 of output on mid-tier and frontier models. Nodus serves open-weight models such as gpt-oss-120b through a per-token inference API (Beta) that prices input, cached input and output separately and reports the charge on every request. The list price is only half the answer: in a 20-turn agent loop we priced, resent history made up 98% of the tokens, and prompt caching cut the session cost by 78%.

It depends on the tier far more than on the provider. Within a tier, the big labs price almost identically. Between the top and bottom tiers, input prices differ by 20 to 90 times and output prices by 20 to over 200 times.

Tier Model (provider) Input per 1M Cached input per 1M Output per 1M
Frontier Claude Opus 5.5 (Anthropic) $4.00 $0.20 $20.00
Frontier gpt-6.2-astra (OpenAI) $8.00 $0.40 $40.00
Mid Claude Sonnet 5.5 (Anthropic) $2.00 $0.10 $10.00
Mid gpt-6.2-sol (OpenAI) $2.00 $0.10 $10.00
Small Claude Haiku 5.5 (Anthropic) $0.10 $0.01 $0.50
Small gpt-6-luna (OpenAI) $0.10 $0.01 $0.50
Open model gpt-oss-120B (Together AI) $0.15 not listed $0.60
Open model DeepSeek-V4-Flash (DeepInfra) $0.09 $0.018 $0.18

Prices checked November 17, 2026, standard tier, from Anthropic’s pricing page (Haiku rates for prompts up to 100K tokens, cache rates for the 5-minute cache), OpenAI’s API pricing (short context), Together AI’s pricing and DeepInfra’s pricing. Batch and flex tiers cost less. Anthropic and OpenAI both list batch processing at half the standard rate.

Output price per million tokens: gpt-6.2-astra $40, Claude Opus 5.5 $20, Claude Sonnet 5.5 and gpt-6.2-sol $10, gpt-oss-120B on Together AI $0.60, Claude Haiku 5.5 and gpt-6-luna $0.50, DeepSeek-V4-Flash on DeepInfra $0.18. Output tokens from a frontier model cost about 33 to 220 times more than from a small or open model.

Two patterns hold across the table. Output costs 5 times input at Anthropic and OpenAI, and 2 to 4 times input on the open-model rows. Cached input costs 5% to 20% of the input rate wherever it is listed.

Why are input, output, cached and reasoning tokens priced differently?

Section titled “Why are input, output, cached and reasoning tokens priced differently?”

Each token type uses a different amount of GPU time. The price follows the work.

Input tokens are processed together in one parallel pass, called prefill. The GPU reads the model weights once and handles thousands of prompt tokens at the same time, so each input token is cheap.

Output tokens are generated one at a time. Every new token needs another pass over the weights and the growing cache of earlier tokens. That serial work is why output costs several times more.

Cached input is a prompt prefix the provider has already processed and stored. A cache hit skips the prefill work, so Anthropic and OpenAI charge 5% to 10% of the input rate for it ($0.10 against $2.00 on Sonnet 5.5). Writing the cache can cost extra: Anthropic lists $2.50 per million for Sonnet 5.5 cache writes, 1.25 times its input rate, and OpenAI lists the same $2.50 as a 30-minute cache write for gpt-6.2-sol.

Reasoning tokens are the model’s hidden thinking. You never see them, but OpenAI’s reasoning guide says they are billed as output tokens. The Nodus inference guide notes the same for its API: reasoning uses the output-token budget even though the reasoning text is not returned. A model that thinks for 2,000 tokens before a 200-token answer bills 2,200 output tokens.

How do you turn token prices into the bill for a workload?

Section titled “How do you turn token prices into the bill for a workload?”

Count each token type per request, multiply by its rate, then multiply by requests. The formula we use:

cost per request = ( uncached input x input price
+ cache writes x cache-write price
+ cache reads x cached-input price
+ (visible output + reasoning) x output price ) / 1,000,000

The hard part is counting input honestly. Chat and agent APIs are stateless, so every turn resends the whole conversation: the system prompt, tool definitions, every earlier message and every earlier answer. Input grows with each turn while output stays flat.

What does a 20-turn agent session actually cost?

Section titled “What does a 20-turn agent session actually cost?”

We priced one agent session ourselves to show how this plays out. The setup: an 8,000-token system prompt and tool definitions, 1,500 new input tokens per turn (tool results and instructions) and 500 output tokens per turn, for 20 turns. We left reasoning tokens out to keep the arithmetic visible.

Turn 1 sends 9,500 input tokens. Each later turn sends the previous context plus the last 500-token answer plus 1,500 new tokens, so turn 20 sends 47,500. The session totals 570,000 input tokens and 10,000 output tokens. That is 57 input tokens for every output token, and input is 98% of all tokens.

On Claude Sonnet 5.5 without caching:

  • Input: 570,000 x $2.00 / 1M = $1.14
  • Output: 10,000 x $10.00 / 1M = $0.10
  • Total: $1.24 per session, with input 92% of the bill

With prompt caching, each turn reads the previous context from the cache and writes only the 2,000 new tokens. Across the session that is 522,500 cached reads and 47,500 cache writes:

  • Cache reads: 522,500 x $0.10 / 1M = $0.052
  • Cache writes: 47,500 x $2.50 / 1M = $0.119
  • Output: $0.10, unchanged
  • Total: $0.271 per session, 78% less

This assumes every turn lands within the cache lifetime, which is 5 minutes in Anthropic’s listed rates. A slow human between turns can miss it.

Cost of a 20-turn Claude Sonnet 5.5 agent session: without caching $1.140 input and $0.100 output; with caching $0.119 cache writes, $0.052 cache reads and $0.100 output. Without caching, resent history is 92% of the bill; caching turns it into the smallest line.

The gap widens every turn. Without caching, turn 1 costs $0.024, turn 5 $0.040, turn 10 $0.060, turn 15 $0.080 and turn 20 $0.100. With caching, turn 1 costs $0.0288 because of the cache write, then turn 5 costs $0.0116, turn 10 $0.0126, turn 15 $0.0136 and turn 20 $0.0146.

Per-turn cost of a Claude Sonnet 5.5 agent session: without caching it climbs from $0.024 at turn 1 to $0.100 at turn 20; with caching it is $0.0288 at turn 1, then $0.0116 at turn 5 and $0.0146 at turn 20. Uncached sessions get more expensive every turn; cached ones stay nearly flat after the first.

The same session on other models, using the rates in the table:

Model No caching With caching
Claude Opus 5.5 $2.48 $0.542
Claude Sonnet 5.5 $1.24 $0.271
Claude Haiku 5.5 $0.0620 $0.0162
gpt-oss-120B (Together AI) $0.0915 no cached rate listed
DeepSeek-V4-Flash (DeepInfra) $0.0531 $0.0155

For DeepInfra we priced cache writes at the normal input rate, because its page lists no separate write price. gpt-6.2-sol and gpt-6-luna list the same input, cached and output rates as Sonnet 5.5 and Haiku 5.5. Per million tokens processed, the uncached Sonnet session works out to $2.14 and the cached one to $0.467.

Which price should you compare when picking a model?

Section titled “Which price should you compare when picking a model?”

Compare the price of the token type that dominates your traffic. A simple rule follows from the formula: divide the output price by the input price to get a ratio R. If your requests send more than R input tokens per output token, input and caching drive your bill. If they send fewer, output does.

At Anthropic and OpenAI, R is 5. Our agent session sent 57 input tokens per output token, so the cached-input rate mattered more than anything else. A long-form writer that sends 1,000 tokens and gets 3,000 back sits on the other side, and output price decides.

So for chat, agents, RAG and coding assistants, check whether caching exists, what a cache write costs and how long the cache lives. For generation-heavy work and reasoning models, compare output prices and count reasoning tokens. If you can wait for results, batch tiers halve the bill at Anthropic and OpenAI.

How do you check per-token prices on Nodus?

Section titled “How do you check per-token prices on Nodus?”

The Nodus inference API is OpenAI-compatible and serves open models such as gpt-oss-120b and gpt-oss-20b. Its console shows Input, Cached input and Output prices in USD per million tokens for each model, and GET /v1/models returns the same rates. We do not repeat them here because they move as the catalog updates. Catalog cards use a reference mix of 3 input tokens per output token, so check the full rate card if your traffic looks like the agent session above.

Three details from the Nodus inference docs matter for budgeting. The accepted rate is fixed for each request, so a price change or a pricier fallback does not raise it. Every successful chat response includes usage.cost and the count of cached tokens. A named endpoint can carry a spending cap:

Terminal window
$ nodus create inferenceendpoint support-bot --model nodus/gpt-oss-120b --max-cost 50

Once requests through it have spent $50, the next one gets 402 BudgetExceeded. The guide’s example call to gpt-oss-20b shows the receipt format: 84 input plus 21 output tokens, charged $0.000015.

How do you convert a GPU-hour price into dollars per million tokens?

Section titled “How do you convert a GPU-hour price into dollars per million tokens?”

Divide the hourly cost by the tokens the GPU serves in that hour:

$ per 1M tokens = hourly price x 1,000,000 / (tokens per second x 3,600 x share of hours busy)

Throughput and utilization dominate this number, not the hourly rate. Our post on whether self-hosting an LLM is cheaper than an API works through break-even volumes with H100 rates from the Nodus pricing page. For a single machine at your desk, see running an LLM locally vs in the cloud.

No. Each model family has its own tokenizer, so the same text can be a different number of tokens. Compare costs on your own prompts with each provider’s token counter, not on words or characters.

Do I pay for input tokens if the request fails?

Section titled “Do I pay for input tokens if the request fails?”

It depends on how it fails. OpenAI’s reasoning guide warns that a response cut off by max_output_tokens can bill input and reasoning tokens without any visible answer. On Nodus, a request that fails with no tokens is Released and not charged.

Does a longer context window cost more per token?

Section titled “Does a longer context window cost more per token?”

Sometimes. Anthropic charges Haiku 5.5 $0.50 per million input tokens above 100K tokens of prompt, five times its shorter-prompt rate, and OpenAI lists separate long-context prices.

How much does a million output tokens cost on a reasoning model?

Section titled “How much does a million output tokens cost on a reasoning model?”

The same as any output token at that model’s rate, but you buy more of them. Hidden reasoning can be several times the visible answer, so budget output by measured total output tokens per request.

More from the blog