LLM Cost Tracking: The Real Cost of an LLM API Call, Per Run

---

Most teams still estimate LLM cost as (input × input_price) + (output × output_price). In 2026 that mental model is wrong on three axes: prompt caching has split input into three sub-buckets, reasoning tokens are billed separately and can silently inflate a $0.10 request to $5.00, and tool definitions plus image inputs are line items a naive estimator misses.^[1] Effective llm cost tracking on a multi-tenant platform requires a per-run attribution ledger — one append-only row per request, every billable component broken out, indexed by tenant × project × model.

What "cost per LLM API call" actually means in 2026

Effective llm cost attribution per call means instrumenting every billable component. Reference tables verified against vendor pricing pages on 2026-06-17.^[2][3][4] OpenAI cost monitoring production starts with the cached-input column: on gpt-5.5 a cached read at $0.50/MTok is 90% off the $5.00 base. Cost per token attribution multi-tenant work fails the moment any column is dropped; cost-aware llm architecture depends on the full breakdown being visible.

Anthropic (verified 2026-06-17)

Model	Base input $/MTok	Cache hit	Output $/MTok
Claude Fable 5	10.00	1.00	50.00
Claude Opus 4.5–4.8	5.00	0.50	25.00
Claude Sonnet 4.5 / 4.6	3.00	0.30	15.00
Claude Haiku 4.5	1.00	0.10	5.00

Anthropic Message Batches discounts all of the above by 50%.^[5]

OpenAI (verified 2026-06-17)

Model	Input $/MTok	Cached input $/MTok	Output $/MTok
gpt-5.5	5.00	0.50	30.00
gpt-5.4	2.50	0.25	15.00
gpt-5.4-mini	0.75	0.075	4.50
gpt-5.4-nano	0.20	0.02	1.25

Google Gemini (verified 2026-06-17)

Model	Input $/MTok	Output $/MTok
Gemini 3 Pro	1.50	9.00
Gemini 3 Flash	0.75	4.50

Gemini charges $14 per 1,000 grounding queries after a 5,000-prompt free tier.^[4] OpenAI cost monitoring production and the equivalent on every vendor must catch these line items or the ledger undercounts.

Seven billable components every llm cost attribution per call system must track: input cache miss, input cache read (~90% off), input cache write (Anthropic 1.25×–2.0× base — the 1-hour write costs more than a cache-miss read), output tokens (3–5× input price), reasoning tokens (billed separately on o-series; the single most dangerous cost vector in 2026 — a $0.10 gpt-5.4 request can hit several dollars in o3), tool tokens, and image inputs.

Illustrative Scenario — Composite: a legal-tech SaaS uses Claude Sonnet 4.6 for "summarize contract" (~3,500 input + 1,200 system + 400 summary). Naive cost ~$0.015/call; with Anthropic prompt caching (5-minute TTL, 80% hit rate), ~$0.010/call — a 33% saving.

The per-run attribution ledger: schema and OpenTelemetry alignment

The data structure that makes llm cost tracking defensible is a per-run attribution ledger — one append-only row per LLM API call, indexed for cost per token attribution multi-tenant roll-up, with every billable component in its own column. llm cost attribution per call lives at this granularity — the ledger is what makes unit-economics auditable. This is the foundation of any cost-aware llm architecture.

Figure 1 — Per-run attribution ledger row schema. Architecture Design for production cost-tracking harnesses — not shipped. Each row in the ledger captures every billable component of one LLM API call, indexed by tenant × project × model, ready for the §4 roll-up views. Pattern synthesized from OTel gen_ai.* conventions (LangSmith, Helicone, Portkey); the column set in §3.1 of the post is illustrative of the recommended design, not a deployed table schema.

The minimum column set, synthesized from OTel gen_ai.* conventions:^[6][7][8]

Column	Type	Purpose
run_id	UUID	Primary key
timestamp	timestamptz	Request start (UTC)
tenant_id	string	Multi-tenant segmentation
project_id	string	Sub-tenant segmentation
feature_id	string	Which feature triggered the call
model	string	Vendor + model identifier
provider	string	openai, anthropic, google, bedrock
input_tokens	int	Total input
input_tokens_cache_read	int	Subset that hit cache
output_tokens	int	Generated text tokens
reasoning_tokens	int	Reasoning chain tokens
cost_input_usd	numeric(12,6)	Dollar cost of input
cost_output_usd	numeric(12,6)	Dollar cost of output
cost_reasoning_usd	numeric(12,6)	Dollar cost of reasoning
cost_total_usd	numeric(12,6)	Authoritative per-run cost
policy_decision	string	Budget guard outcome

Every row is the materialized form of a span carrying gen_ai.usage.input_tokens, gen_ai.usage.cached_input_tokens, gen_ai.usage.reasoning_tokens, and gen_ai.cost.total_usd.^[6] That alignment lets the ledger join to the trace, the trace to logs, and logs to the schema-validated structured output (LLM schema registry pattern).

Two implementation gotchas: streaming without stream_options: { include_usage: true } returns zero usage — a harness that forgets the flag silently logs zero tokens for every streamed request and undercounts cost by the full request volume (Anthropic's equivalent is worse)^[7]; prices go stale — a centralized pricing JSON refreshed on a 24-hour cache.^[8]

Multi-tenant roll-ups: eight standard views

The ledger is write-once, read-rarely. Dashboards are materialized views over the ledger. The minimum view set for cost per token attribution multi-tenant dashboards:^[8][9][10]

1. tenant × day — daily cost per tenant, 90-day rolling. Headline SaaS metric.
2. tenant × model — which model each tenant is burning cost on.
3. project × week — internal chargeback.
4. feature × model × cost-per-call — product analytics.
5. cache hit rate × tenant × day — leading indicator of cache effectiveness.
6. reasoning tokens × tenant × day — early-warning for o-series bill shock.
7. error rate × cost — failed requests that still cost money (retry storms).
8. policy decisions × cost — what the budget gate prevented.

The most operationally important view is the top-N cost contributors table: which 10 requests, features, tenants, or API keys account for 80% of the bill.^[10] A documented reference example: a preview model burning roughly $6.2K/month at 25× the org's blended rate, 98% traced to a single batch-pipeline key.

Budget enforcement: soft caps, hard caps, kill switches

A ledger without enforcement is a confession booth. llm budget enforcement runs on a four-interval budget model: lifetime, monthly, weekly, daily.^[9] Russian-doll structure prevents account-level runaway (lifetime), quiet drift (monthly), sprint-end surprise (weekly), and the 30-minute fire (daily).

Figure 2 — Budget enforcement state machine. Architecture Design for production cost-tracking harnesses — not shipped. Six states (ok → soft_warning → hard_cap_armed → hard_cap_triggered → fallback_model → kill_switch) with a recovery loop from fallback back to ok. Pattern synthesized from OpenRouter Workspace budgets and the §5 four-interval soft/hard-cap model; the BudgetGuard component is illustrative of the recommended design, not a deployed state machine.

The llm budget enforcement state machine has six states: ok, soft_warning (past 80% — alert fires), hard_cap_armed (past 95% — warning escalates), hard_cap_triggered (HTTP 429), fallback_model (cheaper fallback still under cap), and kill_switch (503, on-call paged).

Reference incidents:

• CIO.com / Benchmarkit, October 2025: 85% misestimate AI costs by more than 10%; nearly a quarter misestimate by 50% or more.^[1]
• DN42 agent AWS bill, May 2026: $6,531.30 of AWS in ~24 hours.^[11]
• Single enterprise Claude bill, May 2026: $500M in one month after usage limits were forgotten.^[12]
• Replit agent database deletion, July 2025: $607.70 in extra charges and a destroyed production database.^[13]

Every LLM app needs a kill switch engageable in under 60 seconds by a non-engineer. Soft caps are a measurement instrument; hard caps are a circuit breaker. The failure-mode side of this flow is in the LLM circuit breaker pattern post.

Five cost-aware architecture patterns

A cost-aware llm architecture is deliberate choices that move the dial.

Figure 3 — Cost observability dashboard heatmap. Architecture Design for production cost-tracking harnesses — not shipped. A 12×14 per-tenant × per-day cost heatmap; the highlighted anomaly cell shows a Replit-style retry loop that triggered a kill switch from the §5 enforcement state machine. Pattern synthesized from Portkey and OpenRouter Analytics API; the roll-up view layer is illustrative of the recommended design, not a deployed dashboard.

1. Model selection on the cost/quality frontier. Route to the smaller model when both clear the quality bar. Opus is $5/$25 per MTok; Haiku 4.5 and gpt-5.4-nano are 10–50× cheaper.
2. Prompt caching with measured hit-rate targets. Both vendors give 90% off cached reads.^[14][15] Exceed the 1,024-token minimum cacheable block, stabilize the cache key, and measure cache hit rate. Without instrumentation: 10–30% hit rates and a net cost increase on Anthropic. With instrumentation: 70–90% hit rates and a 40–60% net cost reduction.
3. Batch API for non-realtime workloads. OpenAI Batch API and Anthropic Message Batches offer 50% discounts.^[5][16] Route overnight report generation, bulk summarization, evaluation runs, and embedding refreshes to batch.
4. Reasoning token budgeting. Every reasoning-capable call declares a max_reasoning_tokens budget; the harness enforces it by truncating or by failing fast with policy_decision: kill_reasoning_overrun. A token accounting harness protects the organization from its own enthusiasm for o3.
5. Streaming with usage accounting. Enforce stream_options: { include_usage: true } (OpenAI) or run the tokenizer post-hoc (Anthropic) on every streaming call.^[7] OpenAI cost monitoring production depends on this — forgetting the flag once silently loses 30% of cost visibility.

Cost tracking — design pattern for production harnesses

Architecture Design for — the four components below are drawn directly from the patterns established in §3–§6. They describe an LLM token accounting harness we recommend for production deployments; they are not shipped in our product surface today. Each maps cleanly to a pattern already detailed above; this section simply gathers them into one reference view for the reader building a similar harness, and the names are illustrative:

• Append-only per-request cost ledger — one row per LLM API call, with every billable component in its own column.
• Roll-up view layer — materializes the eight §4 standard views and the top-N contributor table.
• Budget enforcement component — four-interval soft/hard-cap budget model + the §5 state machine; the llm budget enforcement core.
• Cost-aware model router — picks the cheapest model on the cost/quality frontier.

Request flow (illustrative — Architecture Design for, not shipped): router → budget gate → model client → ledger write → roll-up refresh. Runs alongside the session context management design pattern; cost rows flow into the same observability pipeline that powers the output safety stack post-LLM call.

---

If your LLM app is hitting budget surprises and you want a 20-minute architecture review of your current ledger, budget gate, and routing layer, book a 20-minute architecture review.

Sources