How to Cut LLM Output Token Costs: 9 Techniques That Actually Work

The bill is mostly output, and the per-token gap is the reason

On almost every modern API, an output token costs 5-6x what an input token costs. GPT-5.4 is 2.50 in / 15.00 out per million tokens — exactly 6x. Claude Sonnet 4.6 is 3.00 / 15.00 — exactly 5x. GPT-5.5 is 5.00 / 30.00, also 6x. This ratio is not an accident; generation is autoregressive and serial, so each output token is more expensive to produce than a prefill token that the model processes in parallel. The pricing reflects the compute.

Walk a realistic chat turn. A support assistant sends a 1,000-token prompt (system instructions plus the user message) and the model replies with 600 tokens. On GPT-5.4 the input costs 1000 / 1,000,000 x 2.50 = $0.0025 and the output costs 600 / 1,000,000 x 15.00 = $0.0090. Output is 78% of the $0.0115 turn even though the model emitted fewer tokens than it read.

Codegen skews harder. Say a refactoring agent reads 4,000 tokens of context and emits a 3,000-token diff on Claude Sonnet 4.6: input is 4000 x 3.00 / 1e6 = $0.012, output is 3000 x 15.00 / 1e6 = $0.045. That is 79% of the cost in the part of the request you control most directly — how much the model writes. Anything that trims generated tokens, or moves them to a cheaper meter, hits the dominant line item.

One honest caveat up front: if you stuff enormous documents into context on every call and get terse answers back, your bill can be input-dominated instead. Pull your real per-request input/output split from logs before you optimize. The techniques below assume the common case where output dominates; if yours doesn't, start with prompt caching ROI instead.

Lowest effort, immediate payoff: cap output with max_tokens and stop sequences

The single highest-leverage change is a hard max_tokens ceiling on every call. Without one you are billed for whatever the model decides to write, and you are exposed to pathological runaways — a model that loops, re-explains, or pads a list to 65,536 tokens because nothing told it to stop. Every model here has a large max output budget (GPT-5.4 and Sonnet 4.6 both allow 65,536), and that budget is your liability, not your friend, unless you constrain it.

Set the cap to your real P99 answer length plus a margin, not to the model maximum. If 99% of your answers fit in 500 tokens, a cap of 800 protects you from runaways while almost never truncating a legitimate response. Log how often you hit the cap; a high truncation rate means the cap is too tight or the task genuinely needs more room, and you should raise it deliberately rather than leave it at the default.

Stop sequences are the surgical version. If your output contract ends with a known delimiter — a closing ``` fence, an tag, a sentinel like END — pass it as a stop sequence so generation halts the instant the useful payload is done and you are not billed for a trailing paragraph of commentary. This pairs especially well with models that like to add a friendly summary after the thing you actually asked for.

Caveat: a truncated response can be worse than an expensive one if it corrupts downstream parsing. Truncation should fail loudly (detect finish_reason of length/max_tokens) and either retry with a higher ceiling or surface an error, never silently ship half a JSON object.

• Set max_tokens per route from observed P99 length, not the model default.
• Add stop sequences for any output with a deterministic terminator.
• Alert on truncation rate; treat a spike as a regression, not noise.
• Never let a truncated structured response reach a parser unchecked.

Make the model emit data, not prose: structured and schema-constrained output

The cheapest token is the one the model never writes. A lot of output volume is conversational scaffolding — "Certainly! Here is the information you requested:", restated questions, numbered preambles, and a closing offer to help further. None of it survives into your database, but you pay full output rate for all of it.

Two levers compound here. First, instruct verbosity down in the prompt: "Reply with only the JSON object, no preamble or explanation." That alone routinely cuts 20-40% of tokens on extraction-style tasks. Second, and more reliable, use structured outputs / JSON-schema-constrained decoding where the provider supports it. When the model is forced to emit tokens that conform to a schema, it physically cannot wander into prose; the surface area for padding collapses to the fields you defined.

Schema design is itself a cost lever. A schema with {"sentiment":"pos"} costs a fraction of one that returns {"sentiment":"positive","confidence":"high","explanation":"The customer expressed..."}. If you do not consume the explanation field, do not ask for it. Enums beat free text. Short, stable field names beat verbose ones — every key is emitted on every record, so on a million-row batch a 6-character key versus a 16-character key is real money.

Caveat: schema constraints can degrade quality on tasks that genuinely need reasoning room. If you suppress a chain-of-thought field, accuracy on hard items may drop. The fix is to let the model reason into a scratch field you then discard, or use a separate reasoning step — but measure it. Re-run your eval set after tightening any schema; do not assume cheaper is free.

Stream and cut off the moment the answer is complete

Streaming does not change the per-token price, but it changes when you can stop paying. If you consume tokens as they arrive, your application can detect that the answer is complete — the closing brace of a JSON object, the end of the first valid code block, the third bullet in a list capped at three — and close the connection. You are billed only for tokens generated before you disconnect.

This matters most when the model has a habit of continuing past the useful payload: answering, then explaining the answer, then suggesting follow-ups. With a non-streaming call you pay for the whole tail. With streaming plus early termination you pay for the part you read. In practice this overlaps with stop sequences, and stop sequences are cleaner when you have a deterministic terminator — but streaming termination handles fuzzier cases where you decide "done" in application logic rather than on a fixed token.

Be precise about what you are saving. Disconnecting mid-stream stops billing for not-yet-generated tokens; it does nothing for tokens already produced. So streaming is a complement to verbosity control, not a substitute — it limits the damage when a response overshoots, but the bigger win is making the response short in the first place.

The biggest single lever: route the output-heavy step to a cheaper tier

Because output is the expensive meter, the step that generates the most tokens is exactly where a downshift in model tier pays off most. Take that 3,000-token codegen output. On Claude Sonnet 4.6 it costs $0.045. On Claude Haiku 4.5 (1.00 / 5.00) the same 3,000 tokens cost $0.015 — one third. On Gemini 2.5 Flash (0.30 / 2.50) it is $0.0075, a 6x reduction on the output line. On GPT-5.4 mini (0.75 / 4.50) it is $0.0135.

The trap is routing the whole pipeline to a mini tier and eating a quality cliff. The discipline is to route per step. The output-heavy, mechanically-shaped steps — formatting, extraction, summarization to a template, generating boilerplate code from a clear spec — are where small models hold up well and where the token volume lives. Keep the frontier model for the steps that need judgment, and you capture most of the savings without most of the risk.

Pick the cheaper model by the actual shape of your work, not by a leaderboard. The trade-offs per workload are worked out in cheapest LLM by workload, and the specific small-model decision is in Haiku vs GPT mini. For a side-by-side on the exact pair above, see Haiku 4.5 vs GPT-5.4 mini.

Caveat that engineers miss: token counts are not portable across providers. Claude tokenizes English at roughly 0.8x the GPT token count, so a model that looks slightly pricier per token can be cheaper per task, and vice versa. Compare cost per finished request on your own traffic — the token counter and migration simulator exist for exactly this.

Two-pass: cheap model drafts, frontier model only edits

A more aggressive routing pattern: have a cheap model generate the bulk of the output, then pass it to a frontier model whose job is only to review and correct. The economic bet is that the expensive model's output is small — a short list of fixes, a patch, an approval — while the voluminous first draft is billed at the cheap tier.

Concretely: draft a 3,000-token document on Gemini 2.5 Flash (3000 x 2.50 / 1e6 = $0.0075), then have GPT-5.4 emit a 400-token corrections diff (400 x 15.00 / 1e6 = $0.006) plus whatever input it costs to read the draft. Total output spend lands well under generating all 3,000 tokens on GPT-5.4 ($0.045), and you keep frontier-level judgment on the final result.

This backfires in two specific ways. First, if the frontier model decides the draft is unsalvageable and rewrites it wholesale, you pay the cheap draft plus a full frontier generation — strictly worse than one good model. Cap the editor's output and instrument how often it exceeds a small diff budget; a high rewrite rate means the draft model is too weak for the task. Second, you now pay to feed the draft back in as input on the second pass, so the math only works when the edit output is genuinely small relative to the draft. Two-pass is a real win for review-shaped work and a trap for generate-shaped work.

Stop paying twice for the same answer: response caching and dedup

If two requests would produce the same output, generating it twice is pure waste. An application-layer response cache — key on a normalized prompt hash, store the completion — turns repeated questions into a lookup that costs nothing on the LLM meter. This is distinct from provider prompt caching, which discounts repeated input prefixes; here you are eliminating the output generation entirely on a hit.

Where this earns its keep: FAQ-style assistants, classification of recurring inputs, autocomplete, any surface where a head of popular queries repeats. Normalize before hashing (lowercase, trim, strip volatile fields, round timestamps) so semantically identical requests collapse to one key, and set a TTL that matches how fast the correct answer changes. A semantic cache (embedding-based nearest-neighbor) catches near-duplicates a string hash misses, at the cost of an embedding lookup and a real risk of serving a subtly wrong cached answer.

Caveats that bite: never cache personalized, time-sensitive, or safety-relevant outputs without a TTL and an invalidation story, and never cache non-deterministic responses you actually want to vary. Measure hit rate before celebrating — a cache with a 3% hit rate adds infrastructure and complexity for almost no savings, and you would be better off shortening the outputs instead.

Shrink the output contract itself: IDs over objects, deltas over documents

Sometimes the cheapest fix is to stop asking the model to repeat data you already have. If you hand the model a list of products and want it to pick three, do not have it echo the full product objects back — have it return three IDs and rehydrate them in your code. The model emitted ~10 tokens instead of ~300, and you paid output rate on the 10.

The same principle scales up to documents. When a model is editing or extending something, ask for a delta — a diff, a patch, a list of changed fields with their new values — rather than the entire revised document. On a 3,000-token document where the model changed two paragraphs, a delta might be 200 tokens. At Sonnet 4.6 output rates that is $0.003 versus $0.045 to regenerate the whole thing, a 15x cut on that call. This is one of the highest-payoff patterns in agentic and document-editing workloads, where full-document regeneration is the silent default that quietly dominates the bill.

Apply it to references generally: return citation keys not full quotes, row IDs not full rows, function names not full bodies when the caller already has the source. The discipline is to ask, for every field in the output, "do I already have this on my side?" If yes, have the model point at it instead of reproducing it.

Caveat: deltas add application complexity (you must apply patches correctly and handle malformed ones) and can fail confusingly when the model's idea of the base document drifts from yours. Validate that a patch applies cleanly and fall back to a full regeneration on failure. The savings are large enough to be worth the plumbing, but the plumbing is real.

What this guide deliberately leaves out, and the rule that ties it together

Everything above targets the output meter. The input side — provider prompt caching, retrieval trimming, context pruning, compressing system prompts — is a separate lever with its own break-even math, and conflating the two leads to optimizing the wrong number. If your logs show an input-heavy split, go straight to prompt caching ROI and RAG cost architecture instead; those govern input spend. The batch API (50% off both meters, up to 24h async) is also orthogonal and stacks with most of these techniques for non-interactive work.

One rule binds all nine: re-evaluate quality after every change. A capped, schema-constrained, downshifted, two-pass, delta-only pipeline can save 70% and quietly drop your task success rate from 94% to 81% — a trade you may or may not accept, but must never make blind. Keep a fixed eval set drawn from real production inputs and run it after each optimization. Cheaper output that fails more often is not cheaper; it is deferred cost plus a worse product.

Rank your effort accordingly. Caps and stop sequences are an afternoon and protect you from the worst outcomes. Verbosity control and schema tightening are a day and routinely cut a third. Per-step model routing is the largest single lever and the one to invest real eval time in. Two-pass, response caching, and delta contracts are workload-specific power tools — huge in the right pipeline, not worth the complexity in the wrong one. Measure your input/output split first, then attack the techniques in payoff order, validating quality at every step.

Frequently asked questions

Related reading