How to Estimate the Cost of an AI Feature Before You Build It

What a defensible estimate actually looks like

"How much will this AI feature cost?" is usually answered with a shrug or a number pulled from the air. Neither survives the first invoice. The goal here is to replace the shrug with a single sentence a finance partner will accept: "At our expected volume this runs about $1,500/month, with a realistic floor near $1,050 and a ceiling around $2,100, and the biggest lever is output length." That is a range, a unit cost, and a named lever — not false precision.

You can produce that sentence in an afternoon. The work is mechanical once you stop thinking in dollars and start thinking in tokens. Every API charge is two token counts multiplied by two per-token rates, summed over your request volume, then adjusted for caching, batching, and the overhead nobody remembers until it lands on the bill.

Throughout, I carry one concrete example end to end: a customer-support assistant doing 300,000 requests a month, evaluated on Claude Haiku 4.5. The numbers are real (verified 2026-05-27) and the arithmetic is shown, so you can swap in your own buckets and re-run it.

Step 1 — Break one request into four token buckets

A single LLM call is not one blob of text. It is four distinct buckets that behave very differently on cost and on the levers you can pull later. Estimate each in tokens, not characters or words, because tokens are what you pay for.

Do not guess these counts — measure them. Paste a representative system prompt and a real retrieved context into the AI Token Counter and read the actual number. Guessing is the single largest source of estimate error, and it compounds across millions of calls. One caveat: tokenizers differ. Claude encodes English at roughly 0.8x the GPT token count, so a prompt measured against one model is only an approximation for another — see token counting myths for why.

For the support assistant I'll use these per-request buckets: 1,500 tokens of system prompt plus tool schemas, 2,500 tokens of retrieved knowledge-base chunks, 200 tokens of user message, and 350 tokens of generated answer.

• System prompt + tool schemas — instructions, persona, formatting rules, and the JSON schema of every tool you expose. Sent on every call and routinely underestimated; a few verbose tool definitions can silently add 1,000–2,000 input tokens per request.
• Retrieved / context tokens — RAG chunks, conversation history, pasted documents. Usually the largest and most variable input bucket in a retrieval feature.
• User input — the actual question or message. Small and variable.
• Expected output — what the model generates. The most important number to get right, because output is priced 5–6x higher than input.

Step 2 — Multiply by volume, and keep input and output apart

Volume is the multiplier that turns per-request token counts into a monthly number. Estimate monthly requests from a real signal: current ticket volume, daily active users times sessions, or an API call from an existing analogous endpoint. If you genuinely have no signal, bracket it — carry a low and a high volume forward rather than inventing a point estimate.

The non-negotiable rule: model input tokens and output tokens as two separate streams. They have different rates (output is 5–6x input on every 2026 model), and they respond to different levers. A blended price hides which bucket dominates and sends you optimizing the wrong thing.

For the support assistant at 300,000 requests/month: input per request is 1,500 + 2,500 + 200 = 4,200 tokens; output is 350 tokens. Across the month that is 4,200 × 300,000 = 1,260M input tokens and 350 × 300,000 = 105M output tokens. Notice the input stream is 12x larger by token volume — yet, as the next step shows, it does not automatically dominate cost, because the rates differ.

Step 3 — Apply a real 2026 model's rates (the baseline)

Now attach a candidate model and its real per-token rates. Pick the model that clears your quality bar on your own data, not the cheapest one on the table — then separately check whether a cheaper tier also clears it. Haiku 4.5 is a defensible default for a high-volume support assistant: at $1.00 input / $5.00 output per million tokens it sits well below frontier pricing while still handling grounded, tool-using support replies. If quality at this tier is borderline on your data, the same arithmetic runs unchanged on Sonnet 4.6 at $3.00/$15.00; estimate, don't assume.

Compute each stream at standard (non-cached, non-batch) rates first — this is your honest baseline before any optimization.

• Input: 1,260M ÷ 1M × $1.00 = $1,260
• Output: 105M ÷ 1M × $5.00 = $525
• Baseline monthly total: $1,785

The baseline, read closely

That $1,785 baseline already teaches you something a pricing table cannot. Output is only 8% of token volume but 29% of cost ($525 of $1,785). And inside the input stream, the 1,500-token system-and-tools prefix is identical on every call — it alone accounts for 1,500 × 300,000 = 450M tokens, or $450/month, paid over and over for the exact same bytes. That repeated prefix is the obvious first target, and it is precisely what prompt caching attacks.

Before optimizing, sanity-check the baseline against a wrong-order-of-magnitude smell test. If your back-of-envelope had landed at $50 or $50,000 for this workload, you'd know a token count or the volume was off by 10x. $1,785 for 300k grounded support answers is plausible — roughly six tenths of a cent per request before levers. Now we drive it down.

Step 4 — Apply the two big levers, in order

Two levers move real money: prompt caching first, then the Batch API where the workload tolerates async. Apply them in that order because caching is almost always available, while batch is only available when you don't need the answer in real time.

Prompt caching targets the reused prefix. Anthropic caching charges a one-time write premium (Haiku: $1.25/1M to write) and then serves cached reads at $0.10/1M — a 10x discount versus the $1.00 standard input rate, i.e. ~90% off the cacheable slice. OpenAI and Gemini apply cache reads automatically with no write fee; the mechanics differ but the principle is identical (see prompt caching ROI). For the support assistant, cache the 1,500-token system+tools prefix and leave the variable buckets at standard rate.

Caching takes the baseline from $1,785 to $1,380 — about $405/month or 23% off, paid almost entirely out of the repeated prefix. Cache write charges and TTL refreshes add a small overhead, on the order of tens of dollars at this traffic; we fold that into the buffer in Step 5 rather than pretend it's zero.

The Batch API gives 50% off both input and output for work that tolerates up to ~24h latency (see how the Batch API works). A live support assistant is synchronous, so batch does not apply to it — an honest estimate says so. But the batch-eligible cousins of this feature (overnight ticket summarization, backfilling tags, eval runs) should be priced at batch rates: the same 300k-request shape run fully on batch would be 1,260M × $0.50 + 105M × $2.50 = $892.50, exactly half the baseline.

• Cached prefix: 1,500 × 300,000 = 450M ÷ 1M × $0.10 = $45 (down from $450)
• Variable input (2,500 + 200 = 2,700 tokens): 810M ÷ 1M × $1.00 = $810
• Output (unchanged, real-time): $525
• Cached real-time total: $1,380

Step 5 — Add the buffers people forget

The optimized $1,380 is still an underestimate, because production has overhead a clean spreadsheet omits. Add it explicitly so the number isn't quietly low.

For the expected case I add ~3% for retries and ~$80 for cache-write overhead and TTL churn on top of $1,380, which rounds to roughly $1,500/month as the planning number. The spike margin lives in the high end of the range rather than the expected case — you don't want to inflate the everyday number, but you do want the ceiling to absorb a bad month.

• System + tool-schema tokens on every call — already counted here because we put them in bucket one, but this is the most common omission. If you priced only "the user's question and the answer," you undercounted input by the entire prefix. Double-check yours is in.
• Retries and failures — timeouts, 5xx, and tool-call do-overs re-spend tokens. A 2–3% retry rate is typical for a healthy integration; budget it. A flaky tool can push this much higher — watch it in monitoring.
• Spike margin — traffic is lumpy. A launch, a press hit, or a Monday-morning support surge can push a day 2–3x above average. Carry a 15–30% spike margin so a busy month doesn't blow the forecast.

Step 6 — Convert to a range, a unit cost, and a decision

A single number invites false confidence. Express the estimate as low / expected / high, and derive a per-request unit cost that makes go/no-go obvious. Build the range from honest scenario swings, not arbitrary ±percentages: the low assumes shorter answers, fewer retrieved chunks, and a warm cache; the high assumes longer outputs (~450 tokens), more context, elevated retries, and the spike margin firing.

The unit cost is what makes the decision defensible. Half a cent per resolved support interaction is trivially below the loaded cost of a human reply, so this is a clear go. If your unit cost ever approaches the value of the action it performs, that's a no-go signal independent of the monthly total.

Finally, name the single biggest lever to optimize first — here it's output cost ($525 of the optimized total, untouched by caching). That points you at cutting output tokens (tighter response formats, max-token caps, structured output) as the highest-leverage next move, ahead of any further input tuning. If your feature is retrieval-heavy instead, the lever may be context size — see RAG cost architecture.

• Low: ~$1,050/month
• Expected: ~$1,500/month
• High: ~$2,100/month
• Unit cost (expected): $1,500 ÷ 300,000 = $0.005 per request — half a cent per answered ticket

Step 7 — Close the loop with a pilot and monitoring

A spreadsheet estimate is a hypothesis. Validate it cheaply before committing the whole forecast. Run a small real-traffic pilot — mirror 1–5% of production requests (or a few thousand calls) to the real model and log actual input tokens, output tokens, retries, and cache-hit rate. Compare those measured per-request token counts to your buckets. If output is running 450 tokens instead of your assumed 350, you found the gap before it became a 30% bill surprise rather than after.

The two numbers that most often diverge from estimate are output length (models are chattier than you expect) and cache-hit rate (sparse or bursty traffic keeps the cache cold, so you pay writes without earning reads). Both are directly measurable in a pilot and both are fixable. A miss here is not a failed estimate — it's the estimate working as designed, surfacing the variable that needed calibration.

Once live, the estimate becomes a budget only if you watch it. Track spend per day and per request, alert on drift from your expected unit cost, and re-run this whole calculation whenever the prompt, the retrieval depth, or the model changes — any of which silently shifts every bucket. Set that up with LLM cost monitoring, and when a cheaper model tempts you, price the switch first with the migration simulator rather than guessing.

Frequently asked questions

Related reading