LLM Cost Monitoring and Alerting: A Practical Setup

The bill you can't see

A provider invoice tells you one number: what you owe this month. It does not tell you which feature spent it, which customer drove it, or whether yesterday's number is normal. By the time the invoice lands, the money is already gone and the spike that caused it is two weeks old and impossible to reconstruct. The fix is not a fancier dashboard bolted on after the fact. It is logging the right fields on every single request, before anything else, so that every dollar can be traced back to a line of traffic.

Everything in this guide builds on one decision: instrument at the request level, store the token counts (not just the dollars), and tag each request with enough context to slice the spend later. Get that right and budgets, alerts, anomaly detection, and per-customer profitability all fall out of the same data. Get it wrong and you are forever reverse-engineering a monthly PDF.

This is deliberately vendor-neutral. The same schema works whether you call OpenAI, Anthropic, Gemini, or a mix, and it survives the day you add a second provider — which is the day most ad-hoc cost tracking quietly breaks.

What to log on every request

Wrap your LLM client once, and emit a structured event after every call. The event is cheap — a few hundred bytes — and it is the raw material for everything downstream. The non-negotiable fields:

• request_id — a UUID you generate, so you can join this event to your application logs and traces.
• model — the exact model string (claude-sonnet-4-6, gpt-5-4), not a friendly alias. Aliases drift; the billed model is what you priced.
• input_tokens, output_tokens — read these from the API response usage object, never estimate them after the fact.
• cache_read_tokens, cache_write_tokens — the providers report these separately and they are priced separately; folding them into input_tokens destroys your ability to see cache ROI.
• feature — a stable tag like summarize, chat, rerank. The single highest-leverage field for attribution.
• tenant_id / customer id — who this request was for.
• latency_ms — wall-clock time; spikes here often precede cost spikes (retries, long generations).
• retry_count — how many times this logical request hit the API. Runaway retries are a classic invoice surprise.
• finish_reason — stop vs length; a flood of length truncations means you are paying for max-output ceilings.

Why token fields beat dollar totals

The instinct is to log cost_usd and be done. Resist it. The dollar figure is a derived value that bakes in a price you believed was true at the moment of the call. When a provider changes a rate, every historical dollar you stored is now a fiction you can never correct, because you threw away the raw token counts that produced it.

Store the tokens. Compute dollars in the query, against a pricing config (covered below). Then a rate change is a one-line config edit and your entire history re-prices correctly. The token counts are also the only fields that let you answer engineering questions: an output-token spike points at verbosity or prompt regressions, an input-token spike points at context bloat or runaway history, and a collapse in cache-read tokens points at a caching bug. A dollar total tells you none of this — it just goes up.

Concretely: a request to Claude Sonnet 4.6 with 8,000 input and 1,200 output tokens costs 8000 × $3.00/1M + 1200 × $15.00/1M = $0.024 + $0.018 = $0.042. Store the 8000 and the 1200. The $0.042 is something you recompute, not something you persist.

Attribution: feature and customer tags

Aggregate cost is a vanity metric. The actionable view is cost grouped by feature and by tenant_id, because LLM spend is almost always wildly skewed. It is routine to find that 5% of traffic drives 50% of cost — a single power-user tenant running a chat feature against a 200K-token context, or one background job that re-summarizes the same documents every hour. You cannot fix what you cannot name.

Feature tagging answers "where does the money go?" If chat is 70% of spend, that is where optimization effort belongs, not on the title-suggest feature that costs $11 a month no matter how clever you are. Tenant tagging answers a sharper question: who is unprofitable?

Work the unit economics per customer. Suppose a seat sells for $20/month. Tenant A makes 400 requests/month at ~2,500 input + 600 output tokens each on GPT-5.4: 400 × (2500 × $2.50/1M + 600 × $15.00/1M) = 400 × ($0.00625 + $0.009) = 400 × $0.01525 = $6.10. Healthy — 30% of revenue. Tenant B hammers the same feature 9,000 times a month with 12,000-token contexts: 9000 × (12000 × $2.50/1M + 600 × $15.00/1M) = 9000 × ($0.030 + $0.009) = 9000 × $0.039 = $351. That tenant pays $20 and costs $351. Without per-tenant tagging, Tenant B is invisible — averaged into a blended COGS that looks fine while it quietly eats the margin of forty profitable customers.

Once you can see this, you have options: a usage ceiling, a metered overage, routing heavy tenants to a cheaper model, or aggressive caching. But the precondition is the tag. Add it before you need it; you cannot backfill attribution onto requests you already made.

Turning tokens into money: the pricing config layer

Never hard-code rates in application code. A price embedded as a constant is a time bomb: a provider revision silently makes every estimate and alert wrong, with no error, no deploy, no signal — just slowly diverging numbers until the invoice corrects you. Put rates in a config layer you can edit without shipping code: a database table, a JSON file fetched at startup, or a small pricing service.

The shape is simple — keyed by model and rate type: { "claude-sonnet-4-6": { "input": 3.00, "output": 15.00, "cache_write": 3.75, "cache_read": 0.30 } }, all per 1M tokens. Your cost function is then cost = (input_tokens × rate.input + output_tokens × rate.output + cache_read_tokens × rate.cache_read + cache_write_tokens × rate.cache_write) / 1e6. One function, every model, every historical row.

Three rules keep this honest. First, normalize every rate to per-1M tokens on the way in, so you never mix units — a per-1K vs per-1M slip is a 1,000× billing error. Second, version the config and stamp each cost computation with the config version, so you can audit which rates produced a given number. Third, treat the four rate types as first-class: standard input/output, cache read, and cache write are genuinely different prices, and Anthropic's cache-write premium ($3.75/1M write, then $0.30/1M reads on Sonnet 4.6) vs OpenAI/Gemini's no-write-fee model means a blended rate will lie to you.

For the full breakdown of what each rate on a provider page actually means — input/output asymmetry, batch columns, cache lifetimes — see How to Read LLM Pricing. The compare pages such as Claude Sonnet 4.6 vs GPT-5.4 keep current rate cards you can seed your config from.

Budgets, ceilings, and a kill-switch

Alerts should be about rate of change, not just totals, because a monthly threshold fires on day 28 — far too late. Track burn rate: spend per hour or per day, compared against the budget you have left. The useful alert is "at the current daily burn, you will exceed the monthly budget by the 19th," computed each morning from yesterday's tagged spend. That gives you eleven days to react instead of a post-mortem.

Layer three controls, from soft to hard.

The kill-switch needs near-real-time data, which is why per-request events that land in a fast store within seconds matter more than a nightly batch. You do not need exact accounting at this latency; a count of tokens-per-minute by a rough rate is enough to detect a four-alarm fire.

• Burn-rate alerts (soft). Daily and weekly: if today's spend is projected to overshoot the period budget, notify the on-call channel. Per-feature thresholds catch a single feature regressing without the global number moving much.
• Per-tenant ceilings (medium). A monthly token or dollar cap per tenant. At 80% notify the account owner; at 100% degrade gracefully — queue, throttle, or drop to a cheaper model — rather than serving an unbounded bill.
• Hard kill-switch (last resort). A circuit breaker on absolute spend velocity. If cost-per-minute crosses a ceiling that no legitimate traffic could reach, stop making calls and page a human. This exists for the runaway loop — the agent that retries forever, the job that fans out unbounded — where a bug can burn a month's budget in an afternoon.

Anomaly detection without machine learning

You do not need a model to catch cost anomalies. A trailing median per feature catches the vast majority of real incidents and almost never false-alarms. For each feature, compute the median daily spend (or median tokens-per-request) over the trailing 14 or 28 days, then alert when today deviates by more than a set factor — say 2x. Median, not mean, so a single past spike does not poison the baseline.

Watch three signals specifically, because they cover most real-world blowups:

Run this as a scheduled job over your event store. It is a handful of SQL queries and a comparison — no infrastructure beyond what you already have for logging.

• Output-token spikes. The usual culprit. Output costs 5–6x input across the board (GPT-5.4 is $2.50 in / $15.00 out, a 6x ratio; Sonnet 4.6 is $3.00 / $15.00, 5x), so a verbosity regression — a prompt change that makes the model ramble, or a removed max_tokens — hits the bill hard. Alert on median output-tokens-per-request by feature.
• Cache-hit-rate drops. If cache reads as a share of input tokens fall off a cliff, a deploy probably broke your prompt prefix (a timestamp injected into the system prompt, a reordered context block). You silently revert from cache-read pricing to full input pricing — on Sonnet 4.6 that is paying $3.00/1M instead of $0.30/1M, a 10x jump on the cached portion.
• Retry-count spikes. A rise in retries means you are paying for the same logical request two or three times. Often the first sign of a provider incident or a malformed-request bug.

Dashboards that drive decisions

A dashboard earns its place only if a number on it changes what someone does this week. Skip the vanity total-spend gauge and build views that map to levers:

Pair the cost-per-feature view with the work in Estimate an AI Feature Cost so your monitored reality can be checked against the model you planned with. When the dashboard says a feature is 3x its estimate, that gap is the headline, not the absolute dollar value.

• Cost per feature (stacked, over time). Shows where the money is and which feature's optimization will actually move the bill.
• Cost per 1,000 requests, by feature. A unit-economics number that is stable as traffic grows, so you spot efficiency regressions that a raw total hides under traffic growth.
• Cache hit rate (cache_read_tokens ÷ cacheable_input_tokens). Your single best early-warning indicator; a drop here is free money walking out the door.
• Output/input token ratio trend. Creeping upward means responses are getting longer relative to prompts — and since output is the expensive side, this is your margin slowly eroding before any alert fires.

Build vs buy

A self-built setup is genuinely small: a client wrapper that emits the event, a table (or columnar store) to land it, a pricing config, a nightly anomaly query, and a dashboard in whatever BI tool you already run. A focused engineer ships the core in a few days, and you own the data, the schema, and the privacy posture — nothing about your prompts or customers leaves your infrastructure. The cost is maintenance and the fact that real-time alerting and per-trace debugging are extra work you have to do yourself.

Hosted LLM-observability tools give you traces, token accounting, dashboards, evals, and alerting out of the box, with per-request drill-down that is tedious to build well. The tradeoffs are real: you route request metadata (sometimes full prompts and completions) through a third party, pricing often scales with event volume so it can rival the LLM spend you are trying to control, and you inherit their schema and their model-coverage lag when a new model ships.

A reasonable rule: build it yourself when your need is cost attribution and budget control specifically — that is a small, well-defined problem and the data is sensitive. Reach for a hosted tool when you also want deep prompt-level debugging, eval pipelines, and trace visualization across a complex agent, and the engineering time to replicate that exceeds the subscription. Many teams do both: a thin internal cost ledger they trust for finance, plus a hosted tracer for debugging.

Whichever you choose, the per-request fields in this guide are the foundation — a hosted tool that does not capture your feature and tenant tags cannot do attribution either. When the analysis points toward a cheaper model, the Migration ROI Simulator turns your real token mix into a switch-cost estimate.

Frequently asked questions

Related reading