Research-backed calculator guide

AI Agent Loop Cost: Set Max Iterations Without Wasting Completion Rate

Learn how to price an AI agent loop, choose a max-iteration ceiling, and protect completion rate with token, progress, and budget guardrails.

An AI agent loop is not priced by the number of user prompts alone. It is priced by the model and tool work performed on every turn, including the context that may be sent again on later turns. A max-iteration setting is therefore both a reliability control and a cost control.

What a max-iteration setting actually limits

An agent loop usually repeats a cycle: the model receives the task and current state, chooses whether to answer or call a tool, the runtime executes the tool, and the result is added to state for the next model call. LangChain’s current agent documentation describes this as a model calling tools in a loop, surrounded by a harness of prompts, tools, and middleware ^[1]. The word “iteration” is not universal, so define it in your own telemetry.

In this guide, one iteration means one model decision plus the tool work directly triggered by that decision. Some frameworks count model calls. Others count graph steps, tool calls, or turns. If one model message calls three tools in parallel, your billing ledger may record one model iteration but three tool events. Write the definition beside the setting so operators do not compare unlike counters.

The maximum is an unconditional ceiling, not a target. A successful agent should normally stop earlier when it has met its acceptance test. A cap protects the tail: it stops a retry spiral, repeated search, or no-progress state when the model does not reach a terminal answer. It should not be the only guardrail. A run can hit a low dollar limit before its iteration limit, or a single tool call can consume most of the time budget.

Treat the cap as one layer in a control stack:

1. Goal check: stop when the output satisfies a measurable acceptance test.
2. Progress check: stop or change strategy when the state, error set, or completed-work count does not improve.
3. Iteration cap: stop after the maximum number of defined turns.
4. Token and spend caps: stop when the run reaches its input, output, or dollar allowance.
5. Tool and time limits: stop when tool calls, retries, or wall-clock time exceed their limits.
6. Escalation: return a partial result or ask for review rather than silently pretending success.

This layered design matches the practical guidance in the inspected loop literature: hard maximum steps, explicit termination conditions, state tracking, and escalation are more useful than a single retry number. The cap protects money; the acceptance test protects completion quality.

Build the cost model before choosing the cap

Start with a trace sample from one task family. Separate costs that are fixed from costs that grow with the loop:

• Fixed input per call: system instructions, stable tool schemas, and the original user task.
• New input per iteration: a fresh tool result, retrieved document, or user update.
• Repeated history: prior tool results, messages, and any reasoning or state that is resent.
• Output per iteration: visible response, structured tool arguments, and any billed reasoning or output tokens.
• Tool charges: search calls, code execution, retrieval, storage, or other metered services.
• Failure overhead: retries, fallback model calls, and post-run review.

A deliberately simple full-history estimate is:

input_N = N × S + U × N(N + 1) / 2 + R × N(N - 1) / 2

output_N = N × R

Here N is the number of iterations, S is fixed input tokens per call, U is new input or tool-result tokens added per iteration, and R is output tokens per iteration. The second and third terms model a history that is repeatedly sent. They are an estimate, not a provider invoice formula. Your API may use server-side state, caching, compaction, or a different tokenization scheme.

The corresponding token-only estimate is:

cost_N = input_N × input_rate + output_N × output_rate

Use rates in dollars per token, or divide the provider’s dollars-per-million-token rates by 1,000,000. Vendor pricing pages separate input and output units and may list cached-input prices or cache-write prices. Anthropic’s pricing documentation, for example, lists base input, cache operations, and output as separate per-million-token rates and explains that cache reads are priced differently from ordinary input ^[2]. OpenAI’s pricing documentation likewise lists input, output, and cached-input columns and states that tokens are billed at the selected model’s rates ^[3]. These are vendor references, not a promise that every model or deployment has the same rate.

If a provider or gateway returns actual usage fields, prefer those fields over token estimates. Record model name, service tier, region, cache status, input tokens, output tokens, tool units, and currency for every turn. Keep estimates and vendor quotes separate: an assumed rate is a planning input; a published price is a vendor quote that can change.

Worked cost example

Assume a support-triage agent with these planning inputs. The rates below are illustrative, not a quote from a provider:

• Fixed input S: 1,200 tokens per model call.
• New context U: 1,000 tokens added per iteration, such as a tool result or retrieved case detail.
• Output R: 300 tokens per iteration.
• Input rate: $5.00 / 1M tokens.
• Output rate: $15.00 / 1M tokens.
• Tool charges: $0.00 in this isolated example.
• Per-iteration success probability: estimated from an internal trace sample, not assumed to be a universal model property.

The table makes the arithmetic inspectable. “Success by cap” is a hypothetical completion curve for one task cohort, included to show how a cap decision can be evaluated against quality. It is not a benchmark, a fixed per-iteration probability, or a guarantee.

For five iterations, the input total is 5 × 1,200 + 1,000 × 5 × 6 / 2 = 21,000 tokens. Output is 5 × 300 = 1,500 tokens. The resulting token cost is (21,000 / 1,000,000 × $5) + (1,500 / 1,000,000 × $15) = $0.1275.

The eighth iteration adds $0.1245 versus the five-iteration cap in this scenario and raises illustrative completion from 63.3% to 80.0%. The tenth adds another $0.0780 and raises completion by 3.7 percentage points. Whether that tenth turn is worthwhile depends on the value of a successful case, the cost of human fallback, latency tolerance, and the observed quality of the extra completions.

This table excludes search, retrieval, code execution, storage, and retry charges. If a tool costs $0.002 per call and the agent averages one call per iteration, add N × $0.002. If 8% of runs require one full fallback attempt, multiply the fallback run cost by 0.08 and add it to expected cost. Keep those terms visible instead of hiding them inside a single “cost per run” average.

How to select a cap without wasting completion rate

Do not pick a cap from a single successful trace. Partition historical runs by task type and record the iteration at which each run first met the acceptance test. Then compare candidate caps such as 3, 5, 8, and 10 using at least these metrics:

• completion rate by cap;
• median and P95 iterations for successful runs;
• cost per started run and cost per successful completion;
• percentage of runs stopped for no progress;
• percentage of runs stopped by spend, token, time, or tool limits;
• human-review rate and severity of failed outcomes.

The best cap is often not the one with the highest raw completion rate. A late iteration can rescue a small number of difficult cases while adding a large amount of context and latency to every run. Calculate cost per successful completion:

cost per success = expected cost per started run / completion rate

For the worked table, an eight-iteration token cost of $0.2520 and an illustrative success rate of 80.0% imply $0.3150 per successful completion before tools and fallback work. Ten iterations imply $0.3943 under the illustrative 83.7% rate. The higher cap improves completion but is less efficient in this made-up scenario. Replace the assumed rates with your measured cohort data before making a production decision.

Use a margin of safety for changes in task mix. If a new tool produces larger results, if prompts become longer, or if a model changes its output behavior, the same iteration cap can cost more. A weekly or release-based review should compare estimated usage with invoice or usage-export data. The AI API invoice reconciliation calculator can help compare expected line items with actual usage rather than treating the cap as a billing guarantee.

Controls that keep a cap from hurting quality

A low ceiling can cut off legitimate work. Reduce that risk by making each turn more useful rather than simply increasing the number:

• Give tools narrow schemas and return only fields needed for the next decision.
• Summarize or compact old results when the task no longer needs the full history.
• Mark completed subtasks in durable state so the agent does not rediscover them.
• Use a distinct retry budget for transient tool failures; do not spend all reasoning iterations retrying a 400-level error.
• Require an explicit terminal action or structured success object.
• Preserve the partial result and stop reason when the cap is reached.
• Put a human checkpoint before irreversible external actions.

A cost ceiling is easier to operate when it is attached to a useful unit such as tenant, workflow, model, and task type. Monitor both rate and outcome. A fall in cost per run can be a bad result if completion rate falls faster. Conversely, a more expensive run can be justified if it prevents costly human rework. The AI agent budget guardrail and P95 calculator is a useful companion for separating typical spend from tail spend.

The cap also belongs in deployment documentation. State whether it counts model turns or graph steps, whether parallel tools count separately, which limit wins when multiple limits are reached, and what response the caller receives. This prevents an operator from seeing “max iterations reached” without knowing whether the agent stopped safely, returned partial work, or failed an acceptance test.

For a larger planning exercise, connect loop economics to build and operating scope. The AI agent development cost estimator can hold the implementation and run-cost assumptions that do not belong in the per-run formula.

Frequently asked questions

Is a max-iteration limit the same as a token budget?

No. A max-iteration limit counts a control-flow unit that you define. A token budget counts input and output tokens, which can vary widely from one iteration to the next. One large tool result can exhaust a token budget in a single turn, while a short loop can consume many iterations cheaply. Use both limits and log which one stopped the run.

What is a reasonable starting max-iteration value?

There is no reliable universal value. Start from the P95 iteration count of successful runs for one narrowly defined task type, add a small recovery margin, and test the resulting cap against cost per successful completion. If you do not have traces, choose a conservative pilot cap, return partial results when it is reached, and collect evidence rather than assuming that a larger number improves quality.

Why can ten iterations cost more than ten times one iteration?

If each call resends accumulated messages or tool output, later calls contain more input tokens. The total can follow a triangular or otherwise growing pattern rather than a flat per-turn pattern. Caching, compaction, server-side state, and provider-specific billing can change the exact result, so compare the estimate with usage records.

Should retries count toward the agent’s max iterations?

Count them in a budget that reflects the risk you want to control, but distinguish them in telemetry. A transient API retry and a new reasoning step have different causes and may have different charges. A practical design sets a small retry limit per tool, a global tool-call limit, and a separate reasoning or graph-step cap.

What should happen when the cap is reached?

Return a structured stop reason, the partial result, usage totals, and the next safe action. For low-risk tasks, queue a review or ask the user whether to continue. For high-impact actions, do not automatically continue just because the model has not finished. A cap should create a controlled handoff, not a silent failure or an unbounded retry.

Do cached tokens make max iterations irrelevant?

No. Caching can reduce the price of eligible repeated context, but new tool results, uncached content, output tokens, tool calls, latency, and failure risk remain. Check the provider’s cache-write, cache-read, expiry, and eligibility rules. Treat caching as one input to the model, not a replacement for a termination policy.