Research-backed calculator guide

OpenAI Flex Processing: A Cost-vs-Deadline Decision Rubric

Decide when OpenAI Flex processing is worth the lower token rate by comparing deadline tolerance, timeout risk, fallback cost, and workload priority.

What Flex processing changes

Flex changes the processing tier, not the underlying model quality. You request it with service_tier: "flex" on an eligible request. The call remains synchronous. That distinction matters: Flex can be used for a request that your worker is waiting on, while the Batch API is a separate asynchronous submission pattern. OpenAI positions Flex for lower-priority work such as model evaluations, data enrichment, and asynchronous workloads. ^[1]

The trade is operational. OpenAI says the official SDK default timeout is 10 minutes and that Flex requests may need a longer client timeout for lengthy prompts or complex tasks. The SDK automatically retries a 408 Request Timeout twice before raising an exception, but that behavior is not a complete workload policy: you still need a maximum elapsed time, a duplicate-work strategy, and a defined fallback. ^[1]

Flex can also return a 429 Resource Unavailable response when capacity is not available. OpenAI states that a request rejected for this reason is not charged. You can retry Flex with exponential backoff or retry with Standard when completion is more valuable than retaining the lower rate. ^[1] Do not treat every 429 as proof that another attempt is safe. Rate-limit errors, unsupported configurations, and capacity errors have different causes and should be handled separately.

Microsoft’s Foundry guidance makes the same decision boundary explicit for its preview implementation: use Flex for delay-tolerant work, do not assume automatic Standard fallback, and select Standard, Priority, or provisioned capacity when latency or capacity is critical. ^[4] The exact deployment rules differ between OpenAI’s direct API and Azure, so use the vendor documentation for the endpoint you operate.

The deadline-first rubric

Start with the deadline, not the discount. Write down the latest acceptable completion time for one logical job, then subtract downstream processing, queueing, and delivery time. The remainder is the maximum processing budget for the API call and its recovery path.

Use Flex by default for: nightly enrichment, offline classification, evaluation runs, back-office document analysis, report generation, and queues where a bounded delay is acceptable. These jobs can usually be retried or held for a later window.

Use Standard by default for: user-facing chat, synchronous checkout or authorization decisions, interactive agent steps, incident response, and any request with a strict p95 or p99 latency target. A cheaper request that misses the business deadline is not cheaper at the workflow level.

Use Flex with an explicit Standard fallback for: important background jobs with a fixed delivery window. Try Flex first, retry capacity errors within a bounded budget, then send the same logical request to Standard if the remaining deadline is more valuable than the discount. Record the effective service tier in the result ledger.

Do not silently choose Flex for unsupported configurations. OpenAI’s guide says Flex has limited model availability and refers readers to the pricing page for the supported list. ^[1] A request that is not eligible needs a configuration decision, not an unlimited retry loop.

A compact policy is:

Flex is acceptable if deadline_slack >= retry_budget + expected_queue_delay and fallback_cost is within the job budget.

That is a policy test, not a promise about actual latency. Measure your own p50, p95, timeout, 429, and fallback rates by model and workload class.

Worked cost example

Assume a background evaluation sends 1,000 identical logical jobs to o3. Each successful job uses 2.4 million input tokens and 0.6 million output tokens. These are scenario inputs, not a vendor benchmark. The calculation uses the o3 short-context rates listed on OpenAI’s pricing page on November 2, 2026: $2.00 per million input and $8.00 per million output at Standard; $1.00 and $4.00 at Flex. ^[2]

The 8% fallback rate is an illustrative assumption. It is not a published Flex availability rate and should not be copied into a forecast. If all 1,000 jobs complete on Flex, the token-only saving is 50%. If 20% require a successful Standard fallback, the same arithmetic gives $5,760, a 40% saving. If the request is rejected with a capacity-related 429 and never completed, the Flex rejection itself is not charged, but the business still has an incomplete job. That is why completion rate and deadline compliance belong beside token cost.

The general calculation is:

Flex-first expected cost = successful Flex jobs × Flex cost + successful Standard fallback jobs × Standard cost + other charged work.

Then compare it with the all-Standard baseline and with the cost of a missed deadline. “Other charged work” can include a separately billed retry that actually completes, tool calls, or downstream compute. Do not assume a client timeout means no provider charge; reconcile provider usage records for ambiguous attempts and use idempotency or job keys to avoid duplicate logical work.

For a different model or token mix, use the model pricing comparison to compare the current rate card, then validate the request-level calculation with the AI token cost calculator. If you need to reconcile estimates with invoice rows, use the AI API invoice reconciliation calculator.

How to implement the policy safely

Set service_tier to flex only on requests classified as delay-tolerant. Give the SDK a timeout that fits the workload rather than blindly copying a number. OpenAI’s example increases the timeout to 15 minutes, but that is an example configuration, not a universal SLA or recommendation for every prompt. ^[1]

Use bounded exponential backoff with jitter for transient 408, 429, and 5xx errors. Honor Retry-After when present. Track total elapsed time, not merely retry count. A policy such as “two Flex attempts or 90 seconds, whichever comes first” is more auditable than “retry until it works.” The right values depend on the deadline and observed latency distribution.

When fallback is allowed, resend the logical job with service_tier set to Standard or the endpoint’s documented default. Make the fallback explicit. Do not assume Flex will automatically become Standard. Microsoft’s implementation guide specifically warns that no automatic fallback occurs and recommends recording the processed tier, latency, token usage, errors, and fallback rate. ^[4]

Keep a ledger with at least: job ID, model, requested tier, effective tier, input tokens, cached input tokens where applicable, output tokens, attempt number, HTTP status, elapsed time, and final outcome. This prevents a token-rate comparison from hiding a high retry rate. It also makes the estimate easier to reconcile against provider billing; the AI API invoice reconciliation calculator is useful for that estimate-to-actual step.

OpenAI’s pricing page separates Standard, Batch, and Flex sections and lists rates per 1 million tokens. It also warns that supported models and pricing configurations change, so date your assumptions and recheck the page before publishing a budget or changing routing rules. ^[2] Prompt caching can further change input economics, but it should be modeled from observed cache-read and cache-write meters rather than assumed as a universal discount.

Flex versus Batch: do not conflate them

Flex is a service tier on a synchronous API request. Batch is an asynchronous job workflow. Both may reduce token rates for eligible models, but they solve different operational problems. Flex is useful when you want a normal request shape and can tolerate variable response time. Batch is better when you can package work, poll for completion, and accept the batch contract and delivery window.

OpenAI’s cost optimization guide presents Batch and Flex as separate ways to lower cost, alongside reducing requests, minimizing tokens, and selecting a smaller model. ^[3] If your workload can wait many hours and does not need an immediate response, compare Batch rather than assuming Flex is the cheapest possible option. If the worker needs a synchronous answer before continuing, Flex may be the more direct fit, subject to its latency and availability behavior.

FAQ

Is OpenAI Flex processing always 50% cheaper?

No. Eligible Flex token rates can be half of the corresponding Standard rates, but total workflow cost may be higher than the simple token arithmetic after Standard fallback, charged successful retries, duplicate logical work, tools, or downstream processing. The 50% figure is a rate-card comparison, not a guaranteed end-to-end saving. ^{[1] [2]}

When should I use OpenAI Flex processing?

Use it for non-interactive, lower-priority, or asynchronous work that can wait and can recover from variable latency or temporary capacity errors. Examples include evaluations, enrichment, and background document analysis. Keep user-facing and strict-SLO paths on a tier that meets their latency and capacity requirements. ^[1]

What does a Flex unavailable 429 mean?

A capacity-related 429 Resource Unavailable means Flex could not accept the request at that time. OpenAI says such a rejected request is not charged. You can retry with bounded backoff, or use Standard if the completion deadline matters more than the discount. Still distinguish capacity errors from quota or rate-limit errors, and stop after the retry budget. ^[1]

Does Flex automatically fall back to Standard?

No. The application must explicitly retry with Standard when that is acceptable. Microsoft’s Foundry documentation states that Flex has no automatic Standard fallback and recommends an application-level policy. The precise parameter is endpoint-specific, so confirm the behavior for your deployment. ^[4]

Is Flex the same as the Batch API?

No. Flex is a request-level service tier for synchronous Responses or Chat Completions requests. Batch is a separate asynchronous API workflow. Choose based on whether the worker needs a synchronous response and what completion window it can accept. ^{[1] [3]}

Should I increase the API timeout for Flex?

Possibly, if your deadline allows it. OpenAI notes that the official SDK default is 10 minutes and shows a 15-minute example for Flex. Treat those as documentation values, not a promise that a request will finish within that period. Set a bounded timeout from your own workload deadline, and define what happens after it expires. ^[1]

A practical go/no-go checklist

Before routing a workload to Flex, record the model and current rate card date; confirm that the model and request context are eligible; write the latest acceptable completion time; define a timeout and total retry budget; decide whether a capacity error should retry, queue, or fall back to Standard; and create a ledger that records effective tier and usage. Run a pilot on representative prompts and compare deadline compliance, not just average cost.

The right answer to “when should I use OpenAI Flex processing?” is therefore conditional: use it when delay is cheaper than capacity, the job can recover from a failed attempt, and the expected saving survives your fallback policy. If missing the deadline has a material business cost, price that consequence explicitly before selecting the lower token rate.