Research-backed calculator guide

Expected Cost of a Branching Agent Workflow

Learn how to estimate expected cost per run, cost per accepted outcome, and worst-case spend when an AI agent takes conditional model, tool, retry, and review paths.

What an expected-cost worksheet measures

Start by defining the unit. In this guide, one run means one incoming task followed through the workflow until it is accepted, rejected, or exhausted by the retry policy. It is not one model call, one tool call, or one successful task. Define these separately:

• Run cost: spending for one attempt, including every model, tool, runtime, and review meter on that attempt.
• Expected cost per run: the probability-weighted average when different branches are possible.
• Cost per accepted outcome: total expected spending allocated over accepted tasks. Failed attempts and failed tasks remain in the numerator.
• Path cost: the cost if a specific route is taken. This is needed for controls such as a hard budget or a P95 guardrail.

For every step, write down its unit and quantity. Token charges should state input and output units, normally USD per million tokens. Search should be queries or calls per 1,000. Runtime may be CPU-seconds, GB-seconds, session-seconds, or instance-hours. Human work should be minutes multiplied by an hourly rate. A line with no unit is difficult to reconcile against an invoice later.

The official pages illustrate why a provider-neutral sheet is safer than copying one vendor’s labels. AgentCore lists independent capabilities such as Runtime, Browser, Code Interpreter, Gateway, Memory, Observability, and Evaluations, each with its own usage basis or linked service meter. ^[1] Bedrock separates model prices by input and output tokens and publishes multiple service tiers and batch options. ^[2] OpenAI publishes token rates plus separate tool and container charges and states that Batch API inputs and outputs can be discounted relative to standard processing. ^[3] These are factual product observations checked on 2026-11-02, not universal assumptions for every provider.

The branching formula

Let A be the always-on cost. For branch i, let pᵢ be the probability that branch runs and Bᵢ be the full cost of that branch. If the branches are mutually exclusive and exhaustive, their probabilities sum to 1. If they are optional and can coexist, do not force them to sum to 1; use each trigger probability independently.

E[attempt cost] = A + Σ(pᵢ × Bᵢ)

A branch cost must include all local meters. If a retrieval branch makes two model calls and one search call, include both model calls, the search call, any retrieval infrastructure, and any branch-specific reviewer. Do not charge the search call in the always-on section merely because some runs use it.

For retries, use measured data whenever possible. If r is the probability that an attempt is retried and the retry has the same expected cost, a simple estimate is:

E[retry cost] = r × E[attempt cost]

If retry attempts are more expensive, use a separate retry cost and probability. If retries choose a fallback model, make that fallback another conditional path. For a hard cap of k attempts and per-attempt success probability s, the probability of delivery within the cap under an independence assumption is 1 - (1 - s)^k. That assumption is only a model; correlated provider outages, bad inputs, or evaluator errors can make retries costlier and less effective.

Finally, if q is the probability that a started task is accepted after the configured workflow and retry policy:

Cost per accepted outcome = E[total task cost] ÷ q

Do not use a success rate from a different task mix. A branch that improves quality may also add a reviewer or premium model, so measure acceptance and cost together.

Worked cost example

Assume a USD estimate for a support-ticket agent. The prices below are illustrative inputs, not vendor quotes. Token prices are shown per 1 million tokens; tool, runtime, and reviewer amounts are per attempt. The workflow always runs a planner, two ordinary tool calls, a small runtime allocation, and a reviewer. It then takes one of three mutually exclusive branches.

The arithmetic is transparent. The always-on section is $0.0168 + $0.0650 = $0.0818. The branch contribution is (0.60 × $0.0268) + (0.25 × $0.2700) + (0.15 × $0.0476) = $0.0907. Therefore the expected attempt cost is $0.0818 + $0.0907 = $0.1725, rounded to four decimals.

Now assume 12% of attempts trigger a same-cost retry and the measured accepted-outcome rate after the configured retry policy is 88%. The simple retry estimate is $0.12 × $0.1725 = $0.0207, so expected spend per started task is $0.1725 + $0.0207 = $0.1932. Cost per accepted outcome is $0.1932 ÷ 0.88 = $0.2195. At 10,000 started tasks per month, the estimate is about $1,932.20 before tax and before any fixed monthly platform fee.

The path view tells a different story. The premium branch costs $0.3518 when it runs: $0.0818 + $0.2700. If the policy permits one retry on that same route, the two-attempt path is $0.7036. That is not the P95 by itself. To estimate P95, use logged path costs and frequencies, or a simulation that samples branch, retry, token, and tool distributions. Do not call the most expensive branch “P95” unless at least 5% of observed runs reach or exceed it. For a conservative budget cap, it can still be useful to label $0.7036 as a configured worst-case scenario.

This table is also ready for a monthly calculator. Multiply each meter by volume, keep fixed fees separate, and use the same branch probabilities. If a search provider changes from a per-query fee to a monthly minimum, replace only that meter and document the new unit. AWS, for example, currently lists AgentCore Web Search at $7 per 1,000 queries, while OpenAI’s pricing page lists web search at $10 per 1,000 calls; those figures are provider-specific and should never be blended as though they were one universal rate. ^{[1] [3]}

How to gather inputs without false precision

Use traces to count model calls, tool invocations, branch selections, retries, and review outcomes. Measure token usage from the provider response rather than estimating every request from visible answer length. Include system prompts, tool schemas, retrieved context, intermediate messages, and tool results when the provider bills them. If prompt caching or batch processing applies, record the actual eligible requests and the applicable price row. Do not assume a cache hit merely because prompts look similar.

Use a dated rate sheet. Record provider, model, region, service tier, currency, input rate, output rate, cache-read or cache-write rate, tool rate, and effective date. AWS notes that model pricing depends on provider, modality, model, and tier. ^[2] OpenAI’s page similarly distinguishes standard and batch processing and lists cached-input prices separately from ordinary input. ^[3] If the calculator uses an illustrative rate, label it “estimate” and keep the vendor page beside the input; it is not a quote.

Separate fixed costs from variable path costs. A monthly observability subscription, reserved capacity, or platform fee should not be multiplied by every run. Allocate it per run only when you explicitly choose an allocation denominator. Conversely, do not hide reviewer minutes or search fees inside a model-token line. The AI API invoice reconciliation calculator can help map the estimate’s meters to usage and invoice rows after deployment.

Branch probabilities should come from production traces, a representative evaluation set, or a clearly labeled planning assumption. Recheck them after prompt, policy, routing, or tool changes. A 60% retrieval probability is not a permanent property of the agent; it is a workload observation for a defined period and task mix. Keep separate rates for user segments when the workflow behaves differently for each segment.

Expected cost versus guardrails and build cost

Expected value is useful for budgets and unit economics, but it is not a spend limit. A low average can coexist with an expensive tail if a fallback calls a premium model, launches many tools, or waits on a costly runtime. Pair the worksheet with the AI agent budget guardrail and P95 calculator for per-run ceilings, percentile headroom, and enforcement lag.

It is also not a development estimate. Engineering, integration, testing, security review, and maintenance belong in a separate one-time or recurring model. Use the AI agent development cost estimator when the question is whether a workflow can be built and operated, rather than what one conditional run consumes.

AWS’s current AgentCore page is a useful reminder to list infrastructure details explicitly: Runtime can charge for actual CPU and memory consumption, while some managed features have their own usage dimensions; network transfer and storage can be billed separately. ^[1] The worksheet should therefore include “other infrastructure” as a visible line, even when its planning value is zero. A zero is an assumption, not proof that the service is free.

FAQ

Is expected cost the same as the average cost per successful task?

No. Expected cost per run averages the spending of possible paths for a started task. Cost per accepted outcome divides that spending by the probability of acceptance, so failed attempts and failed tasks are allocated across the accepted outcomes. If acceptance is 88%, dividing by 0.88 increases the unit cost; if acceptance is not measured for the same task mix, the result is not reliable.

Should branch probabilities add up to 100%?

Only when the branches are mutually exclusive and cover every possible route. In the worked example, retrieval, premium reasoning, and fallback are three exclusive outcomes, so they sum to 100%. Optional enrichment, logging, or a safety check can run alongside another branch. Model those as independent trigger probabilities instead of forcing them into one exclusive list.

How do I price a fallback or retry?

Give the fallback its own path cost and multiply it by the probability it is selected. For retries, multiply the cost of the retry attempt by the probability that a retry occurs, or calculate the full retry distribution when costs vary by attempt. Include duplicate model, tool, runtime, and review charges. A retry is not free just because the first response was unsuccessful.

Can I call the most expensive route the P95?

No. P95 is a percentile of observed or simulated run costs. The most expensive configured route is a worst-case or maximum scenario unless its frequency is known. Collect path costs and sort them, or run a simulation using measured distributions for branches, tokens, retries, and tool fees. Use the maximum route for a budget cap only when you label it as a cap or scenario.

Which token count belongs in a model-call cost?

Use the provider’s billable input and output counts. Depending on the API, that may include system instructions, tool definitions, retrieved context, conversation history, tool results, and reasoning tokens. Keep cached input, ordinary input, and output as separate meters when the price sheet does. Never infer a billed total from visible answer characters alone.

What should I do when a vendor changes prices?

Create a new dated rate row rather than overwriting the old one. Keep the provider, model, region, tier, currency, units, and effective date. Re-run the scenario under the old and new rows so the change is visible in your budget and invoice reconciliation. The official AWS and OpenAI pricing pages are primary references for their own current meters, not a guarantee about another provider. ^{[2] [3]}

Limitations and assumptions

This method assumes branch probabilities are meaningful for the chosen task population, meter prices are known or explicitly estimated, and retry attempts can be described by a cost and probability. It does not predict quality, latency, provider availability, tax, exchange-rate movement, capacity reservation terms, or the business value of a successful outcome. It also does not make a vendor quote or promise that actual invoices will match a planning estimate.

For correlated failures, variable context growth, cache expiry, rate limits, or branches whose costs depend on previous tool results, use trace distributions rather than one fixed number per path. Keep p50, p95, and maximum observed cost separate. Reconcile actual usage after launch, then update branch probabilities and meter assumptions. The goal is not a deceptively precise number. It is a model that shows which decision, branch, or meter is responsible for the spend.