What an AI agent actually costs to run

Why a loop costs more than a conversation

When you ask a chatbot a question, you pay once: your prompt in, its answer out. An agent works differently. It plans a step, calls a tool, reads the result, and decides what to do next — and on every one of those turns it sends the entire conversation so far back to the model, because the model remembers nothing between calls.

That re-sending is the whole story. Your system prompt is billed on turn 1 and again on turn 30. The tool result from turn 4 is billed on every turn after it. Output grows in a straight line with the number of turns, but input grows much faster, because each turn carries the accumulated weight of every turn before it. Double the turns and you roughly quadruple the input bill.

This is why teams watch per-token prices fall and their invoice rise anyway. The unit got cheaper; the number of units went up by more. Gartner has a name for the same effect at the industry level — the "inference paradox" — and expects cost per agentic workflow to rise more than fivefold through 2028 even as token prices keep dropping.

Note what this is not. It is not waste, and it is not a bug in your code. It is the direct consequence of a stateless API plus a loop, and it applies to every agent framework equally.

A worked example

Take a support-triage agent: it reads a ticket, looks up the customer, checks recent orders, searches a help centre, and drafts a reply. Say a 1,200-token system prompt including tool definitions, a 400-token ticket, tool results averaging 600 tokens, and 150 tokens of model output per step.

Turn 1 sends 1,600 input tokens. By turn 20 it is sending 15,850 per call — not because anyone wrote a long prompt, but because nineteen earlier turns are still attached:

One task, 20 turns, each turn adding a ~600-token tool result and a ~150-token model step.

The finished task bills 174,500 input tokens and 3,000 output tokens. The same question asked once in a chat window would cost about 1,750 — so the agent costs roughly 100 times more, and 98% of it is input.

That last number is the one to remember. Most people's intuition about LLM cost comes from chat, where output dominates and output is the expensive rate. Agents invert it. You are now buying the cheap token type in enormous quantity, and optimising your output length barely moves the total.

The published 5–30× range assumes shorter loops and tighter tool output than this example. A ten-turn agent with 200-token tool results sits comfortably inside it. A twenty-turn agent with verbose JSON does not.

Not all agents are shaped the same

"Agent" covers several designs with very different bills:

• Single-shot tool use. One model call, one tool, one answer. Barely more expensive than chat, and often the right design for a task people reach for an agent loop to solve.
• The plan-act-observe loop. The case above. Cost scales with the square of the turn count, so the turn limit is the most important number in your configuration.
• Multi-agent setups. A coordinator delegating to sub-agents. Each sub-agent runs its own loop with its own context, and the coordinator pays again to read their summaries. Costs multiply rather than add.
• Long-running or background agents. Anything that keeps working while nobody is watching. The risk here is not the per-task cost but the absence of anyone to notice a loop that will not terminate.

Before optimising a loop, it is worth asking whether the task needs one at all. A surprising share of production "agents" are a single tool call wearing a costume.

Where the tokens actually go

Four line items account for most of the surprise, and none of them are visible in the code you wrote:

• Tool definitions, re-sent every turn. A dozen tools with decent descriptions is easily 1,000–2,000 tokens. That is not sent once — it is sent on every call. A 1,500-token tool block across 30 turns is 45,000 input tokens before the agent has done anything useful.
• Tool results pasted back in full. An API returns 4 KB of JSON and all of it goes into the context, including the fields the model will never read. From then on you pay for those fields on every remaining turn. This is usually the largest avoidable cost in an agent.
• Reasoning tokens. On models that think before answering, you pay for the thinking even though it is not shown to you. A step that returns 150 visible tokens can bill several times that, and it is invisible in your own logs unless you read the usage fields in the response.
• Retries. A malformed tool call, a timeout, or a plan that goes nowhere means part of the loop runs again at full price. A retry is a new request; nothing is reused.

Three of these four are input. That is not a coincidence — it is the same compounding effect seen from a different angle.

Price the task, not the token

Cost per token tells you very little once an agent can fail. What matters is the cost of a task that actually worked, and that is the metric the industry has been converging on.

The arithmetic is simple. If a successful task costs $0.04 and your agent succeeds 85% of the time, your real cost per completed task is $0.04 ÷ 0.85 ≈ $0.047 — a 17.6% uplift, assuming a failure costs a full run. In practice failures often cost more than successes, because an agent that is going wrong tends to burn extra turns before giving up, so treat that as a floor.

This changes model comparisons. A model that is 40% cheaper per token but fails twice as often is not cheaper. Track three numbers per task type:

• Median cost per run — not the mean, which a few runaway loops will distort.
• Success rate, judged by whether the task was actually completed, not whether the API returned 200.
• Cost per success, which is what you compare between models and between prompt versions.

Log the token usage your provider returns on every call, tagged by task. Without that you are guessing, and the guess is usually low.

What actually brings it down

Four changes, in the order they usually pay off:

• Cache the stable prefix. Your system prompt and tool definitions are identical on every turn, which is exactly what prompt caching exists for. It only works if that prefix never varies, so keep timestamps, user names and retrieved documents out of the front of the prompt.
• Trim tool results before appending them. Return the five fields the model needs, not the whole payload. Unglamorous, and frequently the single largest win.
• Cap the turns. Set a hard ceiling and a budget per task. An agent with no limit will eventually find a loop and stay in it, and the cost of that is bounded only by your credit limit.
• Route the cheap steps to a cheap model. Deciding which tool to call next rarely needs your most expensive model, even when writing the final answer does.

Each has conditions attached, and the first two have failure modes that quietly cost you money while appearing to work — a cache that never hits because something variable sits at the front of the prompt looks exactly like a cache that is working. The next article in this series covers all four in detail.

A budget you can set today

Before you measure anything of your own, these are the right order of magnitude for the 20-turn task above, at representative rates:

Derived from the token counts above at representative November 2026 rates, before caching and before failures. Check current per-model rates on the price comparison.

The gap between the first and last row is the entire argument for routing. It is also why the turn cap matters: at flagship rates, an agent that occasionally runs 60 turns instead of 20 costs roughly nine times as much on those runs.

Enforce the ceiling somewhere a bug in your loop cannot bypass. A per-key budget at the gateway layer is the usual answer, because it still holds when the application misbehaves — which is precisely when you need it.

The short version

Agents are expensive for a structural reason, not a wasteful one: they re-send their context on every turn, so input tokens compound with the square of the loop length. You cannot price one by looking at a single request, and you cannot compare two models without knowing how often each one finishes the job. Count the turns, measure the success rate, cache the part that never changes, trim what you paste back, and put a hard ceiling on the rest.