How to Reduce AI API Costs — Token Optimization for Agents

The Bill Always Comes

Building an AI agent is cheap. Running one is not.

The prototype phase is intoxicating. You wire up some tools, write a system prompt, watch the agent do something impressive, and think: this changes everything. Then you check your API dashboard a week later and realize the agent burned through $200 doing what a cron job and a regex could’ve handled.

Every tool call is a round trip. Every long context window is a tax. Every retry is a confession that something upstream is broken. And every single one of these costs tokens, which cost money, which come out of your margin.

The agents that survive in production aren’t the smartest ones. They’re the ones whose operators actually tracked the costs, found the waste, and cut it without cutting capability. The rest quietly get shut down when someone notices the bill.

This guide is about not being that person.

Know Your Token Math

If you’re building agents and you don’t know the difference between input and output token pricing, you’re flying blind with the throttle open.

Input tokens are everything you send to the model: system prompt, conversation history, tool results, user messages. Output tokens are everything the model generates back. Output tokens typically cost 3-5x more than input tokens. That matters.

Here’s where it gets expensive: your system prompt is sent on every single API call. Not once. Every time. A 5,000-token system prompt at 100 calls per day is 500,000 input tokens daily just for instructions. At Claude’s Sonnet pricing, that’s real money before the agent even does anything useful.

Do the math on your own setup:

• System prompt size x calls per day = daily instruction tax
• Average tool output size x tool calls per task = context bloat per task
• Average output length x output token price = generation cost per response

Most people who do this exercise for the first time are horrified. That’s the point. You can’t optimize what you haven’t measured.

Model Selection Is Your Biggest Lever

This is the single highest-impact change you can make, and most people ignore it because they default to the best model for everything.

Not every task needs the most powerful model. In fact, most tasks don’t. Here’s the hierarchy:

• Haiku (or equivalent small model) — mechanical tasks. Parsing structured data, formatting outputs, classification, simple extraction. Fast, cheap, good enough. Use this for 60-70% of your agent’s workload
• Sonnet (mid-tier) — most reasoning tasks. Code generation, analysis, multi-step planning, content creation. The workhorse. Use this for 25-35% of tasks
• Opus (top-tier) — complex reasoning only. Novel problem-solving, nuanced judgment calls, tasks where the quality difference between Sonnet and Opus actually matters for the outcome. Use this for 5% of tasks, max

This alone can cut your API costs by 80% or more. I’m not exaggerating. The price difference between Haiku and Opus is roughly 50x. If you’re running Opus for a task that Haiku handles perfectly, you’re lighting money on fire for no reason.

The move: build a routing layer. Classify the incoming task, pick the cheapest model that can handle it, escalate to a bigger model only if the smaller one fails or the task genuinely requires it. This is how every serious production agent works.

Prompt Engineering for Cost

You’ve been taught to write detailed, comprehensive prompts. That’s good advice for accuracy. It’s terrible advice for cost.

Every word in your system prompt is a token. Every token is charged on every call. So prompt engineering for production isn’t just about getting the right output — it’s about getting the right output with the fewest possible input tokens.

Practical moves:

• Cut the preamble. “You are a helpful assistant that…” — the model already knows what it is. Get to the instructions
• Compress tool descriptions. Don’t write a paragraph explaining what read_file does. “Reads a file at the given path. Returns contents as string.” Done
• Ask for shorter outputs. “Respond in under 100 words” or “Return only the JSON, no explanation.” Output tokens cost more than input tokens. Controlling output length is direct cost control
• Remove redundancy. If you say the same thing three different ways for emphasis, the model got it the first time. You’re paying for the repetition
• Structure over prose. Bullet points and numbered lists parse faster (fewer tokens) than the same information written as flowing paragraphs

I’ve seen system prompts shrink from 8,000 tokens to 2,000 tokens with zero loss in output quality. That’s a 75% reduction in your per-call instruction tax. Over thousands of calls, that’s hundreds of dollars.

Caching and Batching

The cheapest API call is the one you don’t make.

Prompt caching is the biggest unlock most people aren’t using. If your system prompt is the same across calls (and it usually is), providers like Anthropic offer prompt caching that lets you reuse the cached prompt at a fraction of the cost. For a 5,000-token system prompt, cached calls can reduce that portion of the cost by 90%. Enable this immediately if you haven’t.

Result caching is the second biggest. If your agent calls a tool and gets a result, and that result doesn’t change frequently, cache it. Don’t call the GitHub API to check the repo structure every single time when it changes once a week. Don’t re-read a config file on every task when it was the same config file five minutes ago.

Batching is the third. If you have ten similar tasks, don’t make ten separate API calls with ten separate system prompts. Batch them into one call where possible. “Process these ten items” is cheaper than processing each one individually because you pay the system prompt tax once instead of ten times.

The pattern: cache what’s static, batch what’s similar, call the API only for what’s genuinely new.

Context Management

Here’s a cost killer that sneaks up on you: conversation history.

Every time your agent makes an API call in a multi-turn conversation, it sends the entire conversation history as input tokens. Turn 1 sends the system prompt. Turn 10 sends the system prompt plus nine previous turns of messages and tool results. Turn 50 sends everything, and you’re paying for all of it.

This is where agents get expensive fast. A single tool call might return 2,000 tokens of output. After 20 tool calls, you’re carrying 40,000 tokens of tool history on every subsequent call — most of which is irrelevant to the current task.

Fixes:

• Summarize old messages. After N turns, compress earlier conversation into a short summary. Keep the system prompt and recent messages intact. Drop the rest
• Truncate tool outputs. If a tool returns 5,000 lines of code, the agent probably only needs 50. Limit what gets stored in conversation history
• Sliding window. Only keep the last N messages in context. Old messages get summarized or dropped entirely
• Selective inclusion. Not every previous message is relevant to the current step. Build logic to include only the messages that matter for the current decision

Active context management is the difference between an agent that costs $0.50 per task and one that costs $5.00. Same agent, same capability, 10x cost difference.

Monitoring and Budgets

You need three things running at all times: cost tracking, hard limits, and spike alerts.

Cost per task. Not cost per day, not cost per month — cost per task. “This agent costs $0.12 to process a support ticket” is actionable information. “This agent cost $400 this month” tells you nothing about where the waste is.

Cost per agent. If you’re running multiple agents, track each one separately. The content agent might be lean and efficient. The research agent might be hemorrhaging tokens on deep web searches. You won’t know until you measure them independently.

Hard limits. Set a maximum spend per task, per agent, per day. When the limit hits, the agent stops. Not “sends a warning.” Stops. An agent without a spending cap is a credit card with no limit handed to someone who doesn’t check the balance.

Spike alerts. If an agent’s cost-per-task suddenly doubles, something changed. Maybe a tool started returning larger responses. Maybe the agent hit a loop. Maybe a prompt change introduced verbosity. You want to know within minutes, not at the end of the billing cycle.

The uncomfortable question you need to ask regularly: is this agent actually saving money? If the agent automates a task that takes a human 10 minutes but costs $2 in API calls, and the human’s time is worth $0.50 for those 10 minutes, the agent is a net loss. Automation isn’t automatically profitable. Do the math.