Research-backed calculator guide

Effective cost per successful completion across providers

Calculate effective cost per successful routed request by combining provider price, success rate, fallback attempts, quality pass rate, and routing overhead.

Why token price is not route cost

Provider price is normally quoted in billing units such as input tokens, output tokens, cached tokens, image units, seconds, or requests. OpenAI’s pricing page, for example, lists separate input, cached-input, and output rates and distinguishes standard processing from options such as Batch and Flex processing.^[3] Anthropic likewise publishes base input, cache-write, cache-hit, and output prices in USD per million tokens.^[4] Those are essential inputs, but they describe an attempt or usage unit, not necessarily an accepted result.

A routed request can consume more than one attempt. A provider may return a timeout, rate limit, malformed tool call, moderation refusal, or output that fails your quality check. The router may then try another provider or model. OpenRouter documents both provider-level fallbacks and model fallbacks. Its provider-routing guide says default routing considers price, throughput, latency, rate limits, reliability, and quality, while its model-fallback guide says a request can move to the next model after an error and is priced using the model ultimately used.^{[1] [2]}

That behavior creates two different cost questions:

1. What does one attempt cost? This is a billing calculation based on usage and the provider’s current price sheet.
2. What does one accepted result cost? This is an outcome calculation that includes failed attempts, fallback probability, validation, review, and any work required to obtain an accepted result.

The second number is the useful unit for routing decisions. It should be calculated from your own logs, not inferred from a vendor’s advertised savings percentage. Keep price-sheet values dated because model names, rates, cache rules, and service tiers change.

Define “successful” before comparing providers

A reliable comparison begins with an explicit outcome contract. A request can be technically successful but operationally unsuccessful. For example, a provider may return HTTP 200 with invalid JSON, an empty tool argument, a citation that does not resolve, or an answer that fails a policy or factuality check.

Use one or more outcome fields in your event ledger:

• Attempt success: the provider returned a response before the timeout and the transport layer accepted it.
• Schema success: the response parses and meets the required structured-output contract.
• Task success: the result satisfies a deterministic rule, such as retrieving the right record or completing a tool action.
• Quality acceptance: a scorer, reviewer, or business rule marks the result acceptable.
• Final completion: the user-visible workflow ends without a retry, manual rework, or unresolved partial state.

For a customer-support classifier, schema success may be enough. For an agent that changes a database or drafts a regulated response, quality acceptance and final completion may be the meaningful denominator. Do not silently mix these definitions between providers. A route with a high transport success rate may have a lower accepted-output rate.

Log at least the request ID, route policy version, provider, model, attempt number, input and output usage, cache status, latency, error class, fallback reason, quality result, and final outcome. Anthropic’s token-counting documentation notes that preflight counts are estimates and that actual usage can differ slightly; it also recommends counting against the model you plan to use.^[5] Use preflight counts for planning, then reconcile against actual usage records.

The effective-cost formula

For a two-route cascade, define these variables for one original request:

• C1: cost of the primary attempt.
• S1: probability the primary attempt is technically successful.
• Q1: probability a technically successful primary output passes your quality rule.
• C2: cost of the fallback attempt.
• S2: technical success probability of the fallback when invoked.
• Q2: fallback quality-pass probability when technically successful.
• G: gateway, routing, or orchestration cost charged once per original request.
• E: evaluator or validator cost per attempt.

The expected number of attempts in this simple cascade is 1 + (1 − S1). The expected cost per original request is:

Croute = C1 + (1 − S1) × C2 + G + [1 + (1 − S1)] × E

The expected accepted-completion probability is:

Paccepted = S1 × Q1 + (1 − S1) × S2 × Q2

So:

Effective cost = Croute ÷ Paccepted

This formula assumes the fallback is attempted only after a technical failure and that the fallback’s success and quality probabilities are measured for the traffic that reaches it. If you escalate after a quality failure as well, add that branch explicitly. If you retry the same provider, model the retry probability and retry cost as another conditional branch. If a failed attempt is not billed, set its cost to zero only when your provider’s usage record confirms that treatment.

The formula is an expected value, not a promise about every request. Report confidence intervals or a sample size when you use measured probabilities. A small sample can make a provider with a seemingly excellent success rate look artificially cheap or expensive.

Worked cost example

Suppose a team is comparing a low-cost primary route with a more expensive fallback. The figures below are illustrative assumptions for one original request. They are not vendor quotes and do not predict the performance of any named provider.

The tempting but incomplete answer is $0.002, the price of the primary attempt. That ignores fallback spend, gateway overhead, validation, and rejected outputs. Under these assumptions, the cascade costs about $0.004155 for each accepted completion. The expected cost is lower than sending every request directly to a hypothetical fallback-only route at $0.013999 per accepted completion under the same gateway, validation, and quality assumptions. That comparison is useful only because the assumptions and success definition are held constant.

At 100,000 original requests, the expected accepted completions would be 93,852 and expected route spend would be $390, before taxes, minimum commitments, support fees, or other infrastructure not included in the model. The calculation is not a forecast of volume or savings. It is a scale-up of the stated illustrative probabilities and costs.

How to measure provider reliability-adjusted cost

Start with a fixed evaluation window, such as a week or a release cohort, and keep the traffic mix comparable. Split results by task type. A provider may be inexpensive for extraction but costly for long-context planning because its output length, fallback rate, or quality rework differs.

For each route, calculate:

• Attempt cost: actual billed input, output, cached, tool, image, audio, or request units multiplied by the dated rate card.
• Fallback cost: conditional spend from every additional attempt, not just the final provider’s charge.
• Acceptance rate: accepted completions divided by original requests, with the acceptance rule documented.
• Cost per accepted completion: total attributed cost divided by accepted completions.
• Fallback rate: fallback attempts divided by original requests, split by technical error and quality escalation.
• Tail cost: p95 or p99 cost per original request and per accepted completion, so rare loops do not disappear inside an average.

OpenRouter’s routing documentation provides a useful conceptual distinction: price is only one signal alongside throughput, latency, rate limits, reliability, and quality.^[1] Your own ledger should preserve those dimensions instead of optimizing a single average. A route that is slightly more expensive but has fewer failed attempts may have a lower effective cost and a better user experience.

When a price sheet changes, preserve the effective date and rate-card version in your calculation. OpenAI separates input, cached input, and output billing and notes that its API is billed separately from ChatGPT subscriptions.^[3] Anthropic documents cache writes and cache hits as distinct operations and notes that token counts depend on the model tokenizer.^{[4] [5]} These differences make a provider-normalized token assumption unsafe unless you verify usage fields and rates for each provider.

For planning monthly spend, use the AI Agent Cost Calculator | Model Full Monthly Spend after you have estimated cost per accepted completion and expected request volume. For token-level assumptions, see the AI token cost calculator. If the main decision is rate-card comparison rather than reliability, use the model pricing comparison. The LLM cost calculator is useful for checking input/output mix before you add fallback probabilities.

Common mistakes and limitations

Dividing by provider successes instead of original requests. If you count only the requests that reached a provider, you hide how often the route failed before that provider ran. Keep the original request as the cohort key.

Treating HTTP success as business success. Transport success is not the same as valid output, correct tool use, or accepted work. Store separate flags so the denominator can be changed without re-running the entire experiment.

Ignoring conditional costs. A fallback that runs on 8% of traffic still matters at scale. Retries, evaluators, moderation checks, retrieval calls, and human review can also be conditional costs.

Using vendor averages as your own rate. Product pages and case studies may describe a particular workload, date, or customer. Use them as context, not as a substitute for your own usage export.

Comparing unlike traffic. A provider tested on short, easy prompts should not be compared with another provider handling long context or tool-heavy agent turns. Stratify by task, input size, output length, and policy constraints.

Over-reading a small sample. Reliability is a proportion with sampling uncertainty. Keep the raw numerator and denominator, report the window, and avoid declaring a route superior after a handful of requests.

Leaving out non-token meters. Search, image, audio, tool, container, and platform charges may not fit a simple token formula. Add each meter explicitly or label it as excluded. State whether taxes, reserved capacity, gateway subscriptions, human review, and engineering time are outside the calculation.

FAQ

Is effective cost per successful request the same as cost per API call?

No. Cost per API call usually describes one billed attempt. Effective cost per successful request divides the full expected spend for the original request, including conditional attempts and overhead, by the probability of reaching the defined success condition.

Should I count a failed provider attempt if the fallback succeeds?

Yes, if the failed attempt generated billable usage or consumed paid infrastructure. The fallback’s success does not erase the primary attempt’s cost. If the provider confirms that a particular failure incurs no charge, document that exception and set that branch’s cost to zero.

What success rate should I use for an LLM route?

Use the success definition that matches the workflow’s value. Track technical, schema, task, quality, and final-completion rates separately when possible. For a structured extraction tool, schema-valid output may be sufficient; for a customer-facing agent, accepted final completion may be the more meaningful denominator.

How do retries change the calculation?

A retry adds a conditional branch. Multiply the retry cost by the probability that the retry is invoked, then add its success probability to the accepted-outcome equation. Include retries caused by timeouts, rate limits, malformed output, and quality failures if each one can trigger another paid attempt.

Can the cheapest provider still be the best route?

Yes, when it meets the required acceptance rate and its conditional costs remain low. But the cheapest list price is not enough evidence. Compare cost per accepted completion alongside p95 latency, fallback rate, quality, privacy constraints, and operational overhead.

How often should I recalculate effective cost?

Recalculate when a provider rate card, model version, cache policy, routing policy, prompt, task mix, or acceptance rule changes. For active production traffic, keep a rolling view and a release-level comparison so gradual drift in fallback or quality rates does not hide inside a monthly average.