Research-backed calculator guide

Gateway versus direct provider API: price the operational layer before migrating

Compare LLM gateway vs direct API total cost with a transparent model for token spend, gateway fees, infrastructure, labor, retries, and migration risk.

What belongs in LLM gateway versus direct API total cost?

Start with the billable model usage. Providers usually meter input and output separately, with additional units for cached tokens, audio, images, tools, or requests depending on the model. OpenAI’s pricing page, for example, presents separate input, cached-input, and output rates and lists non-token units such as minutes and web-search calls. ^[1] OpenRouter likewise says model and provider prices can differ by input, completion, reasoning, image, or request units, so a comparison that uses only “tokens per month” can be incomplete. ^[2]

Next, price the integration surface. A direct path may have one SDK and one credential at the beginning, but costs grow when the application needs provider-specific retries, rate-limit handling, schema normalization, dashboards, budget alerts, redaction, and a second provider. A gateway centralizes some of that work, but it does not make the work free. It moves the boundary to a platform service that must be configured, secured, monitored, upgraded, and sometimes staffed.

Include the gateway’s commercial model separately from provider usage. Some gateways pass provider rates through and charge when credits are purchased; OpenRouter documents pass-through inference pricing and a separate credit-purchase fee, with plan-dependent terms for bring-your-own-key usage. ^[2] Treat the current vendor terms as a quote that can change, not as a permanent property of the gateway category. If the gateway adds a per-token markup, a subscription, a per-request fee, a log or seat tier, or payment-processing charges, put each line in its own cell.

For a self-hosted gateway, “free software” is not a zero-cost assumption. Add container or VM cost, high-availability capacity, storage, network egress where applicable, secrets management, upgrades, incident response, and the engineer-hours required to operate it. For a managed gateway, replace those lines with the published plan and the internal time to configure and review the service. Either way, retain the estimate as an estimate unless the vendor has given you a written quote.

Observability is part of the cost model, not just a dashboard preference. A cost ledger needs usage, model, requester or team, and a price version that can be reconciled to the provider invoice. Langfuse documents that cost can be ingested from the provider or inferred from model definitions, and that usage buckets must be mutually exclusive to avoid double counting. ^[3] That is a useful control for either architecture: if cached or reasoning tokens are counted incorrectly, the apparent gateway savings may simply be a measurement error.

Finally, account for financial ownership. The FinOps Framework describes technology-cost accountability as a collaboration between engineering, finance, and business teams, with data that should be accessible, timely, and accurate. ^[4] A gateway can make attribution easier, but only if requests carry stable metadata and the data can be exported and reconciled. A single aggregate gateway dashboard is not automatically a finance-ready allocation model.

A break-even test for the operational layer

Let U be monthly provider usage for the matched workload. Let D be direct-path operating cost, excluding that provider usage. Let G be gateway fees and gateway infrastructure. Let O be gateway operating labor and observability. Let M be monthly migration amortization. Let R be the expected monthly cost of retries, fallbacks, and failed work.

The gateway is lower-cost when:

U + G + O + M + R_gateway < U + D + R_direct

The shared usage term cancels when both paths send the same successful requests at the same provider rates. The practical break-even question is therefore:

gateway savings required = G + O + M + R_gateway - D - R_direct

If routing or caching changes the number of provider tokens, do not cancel U. Model the changed workload explicitly. If a gateway routes 30% of easy requests to a cheaper model, that is a usage-mix benefit, not evidence that the gateway itself has a lower provider price. Likewise, if fallback causes duplicate completions, the gateway path may increase usage even while improving availability.

Use three scenarios rather than one point estimate. In a low-volume case, fixed subscription and operations dominate. In a base case, use measured request counts, tokens, error rates, and labor. In a high-volume or incident case, stress the model with a higher retry rate, an outage window, or a second provider. Report ranges when the inputs are uncertain.

Worked cost example

Assume a US-based product sends 1,000,000 successful text requests per month. Each request averages 2,000 input tokens and 500 output tokens. That is 2,000 million input tokens and 500 million output tokens per month. For an illustrative rate, use the GPT-6 Luna prices displayed on OpenAI’s pricing page when checked on November 2, 2026: $0.10 per 1 million input tokens and $0.50 per 1 million output tokens. This is a time-stamped vendor-page observation, not a promise that the rate will remain unchanged. ^[1]

The provider portion is therefore (2,000 × $0.10) + (500 × $0.50) = $450 per month. The rest of the table contains planning assumptions. They are not vendor quotes and should be replaced with your invoices, payroll burden, cloud bill, and incident data.

Under these assumptions, direct is cheaper by $6.50 per month. That is not a universal result. It says the gateway needs either slightly lower operations cost, more avoided incidents, better attribution, or a usage-mix benefit to cross break-even. If the gateway reduces engineering and on-call allocation by another $100 per month, the gateway total becomes $959.50 and saves $93.50. If it causes a 10% increase in provider usage through retries or fallback, add that usage cost before deciding.

The table also shows why quoting only the gateway fee is misleading. The provider spend is $450 in both paths, while the non-provider lines determine the decision. For another model, recalculate the input and output rows with current rates using the model pricing comparison and normalize units with the AI token cost calculator. For measured production traffic, reconcile the estimate against the AI API invoice reconciliation calculator.

How to collect the inputs without guessing

Export 30 to 90 days of request data and group it by model, provider, service, team, and outcome. Capture input, output, cached, reasoning, audio, image, and tool units when the provider exposes them. Record the number of attempts per logical request, not just successful responses. A gateway can make those fields easier to centralize, but a direct implementation can also emit them if the application owns a consistent usage schema.

Measure operational time in hours per month. Ask who handles provider changes, quota incidents, schema drift, cost questions, and dashboard maintenance. Multiply hours by a loaded internal rate or a finance-approved planning rate. Do not mix a cash vendor invoice with an unpriced engineering estimate and call the result precise.

For migration, count discovery, adapter work, prompt and tool-call regression tests, shadow traffic, rollout, rollback, and documentation. Amortize the one-time work over the period in which you expect to benefit. If the gateway is still experimental, show migration as a range rather than hiding it in a zero line.

For reliability, estimate the cost of a failed logical job, not merely the gateway’s uptime percentage. Include reprocessing, customer support, delayed revenue, and manual recovery where those are material. Do not assume a gateway guarantees failover: the provider mix, routing policy, timeout, retry budget, and fallback model all affect the result.

When direct APIs are usually the better financial choice

Direct calls usually fit a single-provider proof of concept, a small team, a low-risk background workload, or a service where the gateway would not remove duplicated controls. They are also easier to reason about when the organization has strict data-path requirements that a third-party gateway cannot satisfy. “Direct” still requires budget alerts, usage logging, key rotation, retry limits, and invoice reconciliation; it is not an exemption from operational controls.

A gateway becomes more financially credible when several teams need the same provider abstraction, when provider-specific behavior is spreading through product code, when attribution is a budget requirement, or when fallback and routing are already being reimplemented in multiple services. The benefit may show up as avoided labor and reduced incident exposure rather than a lower line-item API rate.

FAQ

Does an LLM gateway always cost more than a direct provider API?

No. The gateway may add a fee or service cost, but it can reduce duplicated engineering, observability, retry, and migration work. Compare full monthly totals for the same successful workload. If the gateway only adds a paid control plane and does not remove a meaningful burden, direct calls may remain cheaper.

How do I calculate direct provider API break-even?

First calculate matched provider usage from input and output units. Then compare the gateway’s incremental fee, infrastructure, labor, migration amortization, and expected retry or outage costs against the direct path’s equivalent controls. The gateway breaks even when its incremental total is lower than the direct path’s avoidable cost. Keep token savings from routing or caching as a separate scenario because they change usage rather than simply moving costs.

Should gateway fees be modeled as a token markup?

Only if the gateway actually charges a token markup. Some services pass through provider inference prices and charge a credit-purchase fee, subscription, request fee, log tier, or BYOK fee instead. OpenRouter’s documentation describes pass-through inference pricing and separate fee terms, so inspect the current pricing and billing rules for the service you are evaluating. ^[2] Put each fee in a separate row so a change in one term does not invalidate the entire model.

Is self-hosting a gateway free?

The license may be free, but operation is not. Include compute, high availability, storage, networking, secrets, upgrades, support, monitoring, incident response, and the engineering time to maintain the deployment. If the gateway is a low-volume internal service, a modest cloud bill may still be outweighed by the people cost; if the platform team already operates the required stack, the incremental cost may be small. Measure rather than assume.

What usage data should a gateway or direct integration record?

Record a logical request ID, timestamp, model, provider, input and output units, cached and reasoning units when applicable, status, retry count, route, team or product metadata, and the price version used for calculation. Langfuse’s documentation is a useful reminder that usage buckets must be mutually exclusive and that inferred prices should be versioned or overridden when the provider’s response contains authoritative usage and cost. ^[3]

Can a gateway’s failover savings be treated as guaranteed revenue protection?

No. Failover is a scenario to measure, not a guaranteed outcome. A fallback may have a different quality, price, latency, context limit, or tool behavior. Estimate the probability of an incident, the fraction of traffic affected, the recovery path, and the business value of a successful fallback. Then test the policy on production-like traffic with bounded retries and explicit quality checks.

Limits and update policy

This article’s worked example uses one illustrative model rate observed on November 2, 2026 and several clearly labeled planning estimates. Provider prices, gateway fee schedules, model availability, and billing units can change. Re-run the calculation when a provider changes rates, when a gateway changes its commercial terms, when traffic mix changes, or when a new fallback path is added. The example excludes taxes, foreign-exchange costs, legal review, data-residency premiums, model-quality impact, and the value of new features unless you add them as explicit rows.

The SERP benchmark behind this guide found recurring architecture explainers and gateway comparisons, plus adjacent pricing and community formats. It did not establish search volume, ranking difficulty, competitor absence, future traffic, or a guaranteed business outcome. The useful decision is the one supported by your request ledger, current vendor terms, measured operations time, and a documented sensitivity range.