Cost-per-task, token optimization, ROI frameworks, and the unit economics that determine whether your agent makes money or burns it
Imagine running a brilliant employee who bills by the word — every sentence they read, every word they write, every question they ask costs money. Then imagine they also have to call their colleagues 5 times for every task, those colleagues also bill by the word, and the whole team sometimes goes in circles for 20 rounds before finishing a task. That is what running a production AI agent looks like. Understanding and controlling this cost structure is the difference between an agent product that scales profitably and one that quietly burns cash.
The fundamental equation: Total Agent Cost = Token Cost × Multiplier + Infrastructure + Human Oversight. The token cost is visible and easy to measure. The multiplier — driven by agentic looping, multi-LLM orchestration, error recovery, and verification — is invisible until you look. Human oversight (review, error correction, monitoring) often outweighs token spend by 20–200×. A team that optimizes only token costs while ignoring these other factors is optimizing the wrong thing.
LLM inference costs have declined approximately 5–10× per year since 2022 — a pace even faster than Moore's Law. GPT-4 quality inference that cost ~$20/million tokens in late 2022 now costs ~$0.40–2.00/million tokens. This price compression is driven by three overlapping forces: hardware efficiency gains, algorithmic improvements (~3× per year isolated), and competition between frontier labs.
The single most dangerous economic fact about agents: agentic systems cost 5–25× more per task than direct chat completions doing the same job. A $0.01 chat API call becomes $0.10–0.50 in an agentic workflow. Why?
Every step re-injects the full context — including tool results, prior reasoning, and the system prompt — so token count grows quadratically with the number of turns. A 10-turn agent loop where context doubles each time produces roughly 1,000× the token count of a 1-turn call. Context window management (summarization, compaction) is therefore a core cost-control lever, not just an engineering convenience.
Three compounding effects engineers often miss: (1) Failure recovery — when a tool call fails, the agent retries with more context, escalating both token count and latency. A 10% tool failure rate with retry logic adds 20–40% to average task cost. (2) Verification overhead — well-designed agents check their own work, adding 1–2 extra LLM calls per task. (3) Multi-agent fan-out — orchestrator + 4 subagents each with their own system prompt multiplies token spend before a single tool is called.
Cost optimization for agents is a portfolio problem — each lever has different effort, impact, and risk profiles. Engineers who ship production agents by 2026 typically implement all five in sequence:
The metric that governs whether an agent product is economically viable is reliability-adjusted cost-per-successful-task (RACPST). Many teams optimize cost-per-token while ignoring the other variables that make or break agent economics:
RACPST = (Token Cost + Infrastructure Cost + Human Oversight Cost) / (Task Volume × Success Rate)
A 60% cheaper model that reduces task success rate from 85% to 70% increases RACPST by 6%. A 90% success rate agent with premium model costs less per successful outcome than a 65% success rate agent with a cheap model, even if raw API spend is 40% higher. Build ROI cases on successful outcomes, not on raw API spend.
The human oversight ratio is the most commonly underestimated term. In enterprise deployments, the cost of humans reviewing, correcting, and auditing agent outputs typically runs 20–200× the inference spend. A task that requires a human correction 10% of the time, where each correction takes 15 minutes at $80/hour loaded cost, adds $2.00 in human cost per task — which often exceeds the entire API cost.
Standard ROI calculation for replacing a human workflow: a sales intelligence agent saving 10 hours/week across 15 sales reps recovers ~$15,000/week in productive time (at $100/hour loaded cost), which pays back a $150,000 development investment in 10 weeks. The agent's total inference cost at $500/week is not the governing constraint — the payback math works even at 10× that token spend.
| Platform | Cost Architecture | Key Optimization Feature | Gotcha |
|---|---|---|---|
| Claude / Anthropic | Input: $3/M, Output: $15/M (Sonnet 4.6). Cached input: $0.30/M. Extended Thinking billed as output tokens. | Prompt Caching (prefix): cache blocks of 1,024+ tokens, instant 90% discount on cached reads. Critical for agents with large system prompts or shared documentation. | Extended Thinking tokens are expensive. A thinking budget of 10,000 tokens at $15/M adds $0.15 per call. Set explicit budgets or use standard mode for routine tasks. |
| OpenAI / GPT-4o | GPT-4o: $2.50/M input, $10/M output. Mini: $0.15/M input, $0.60/M output. Context caching available via cached_tokens in API response. | Automatic prompt caching on repeated prefixes. Batch API (50% discount for async processing). Structured Outputs (JSON schema) reduces output verbosity significantly. | o3 and reasoning model costs can reach $60+/M output tokens. Teams often forget that chain-of-thought reasoning on these models produces large token volumes before the final answer. |
| Google Gemini | Gemini 1.5 Flash: $0.075/M input, $0.30/M output. Ultra: higher tier. 1M+ context window is available but very long contexts cost proportionally more. | Context caching via explicit cache creation API — pay once to create a cached context, then use it repeatedly at cache hit pricing. Particularly cost-effective for agents with large knowledge bases loaded per session. | The 1M-token context window is a feature and a trap. Naive implementations stuff everything into context and pay full price. Cost-aware design uses selective retrieval into the large context rather than loading everything. |
| OpenClaw (Claude Code) | Consumes Anthropic API tokens. No additional per-seat token markup. Cost is pure API spend at your Anthropic account pricing, visible in Anthropic console. | Context compaction is automatic — when approaching the context window limit, Claude Code summarizes prior conversation and tools into a compact representation. Dramatically reduces token usage on long coding sessions. SKILL.md patterns enable cached shared context across sessions. | Agents that spawn many subagents (hub-and-spoke) multiply token costs because each spoke has its own full context. Monitor with /cost command in sessions. The 9 CVEs in the ecosystem (covered in Day 14) mostly involve agents making expensive unintended external calls. |