Research-backed calculator guide

Azure OpenAI Spillover per Request: Header Policy, Cost and Fail-Fast Routing

A practical Azure OpenAI spillover request header policy for choosing PTU or Standard routing, measuring overflow, and estimating the separate hourly and token costs.

What the request header actually does

A provisioned deployment reserves model-processing capacity. PTUs are billed for deployed capacity rather than token consumption, so the hourly baseline exists even during quiet periods. Standard deployments use pay-per-token billing. ^[2] The header tells the service which matching Standard deployment may receive the request if the provisioned attempt returns a qualifying non-200 response.

The key header is:

x-ms-spillover-deployment: my-standard-deployment

The primary deployment is still tried first. Microsoft documents spillover when the provisioned request returns a non-200 response such as 429 for exhausted PTUs, 400 for an unsupported long-context request in the documented model cases, or 500/503 for a server error. ^[1] That ordering matters: adding the header does not mean the request bypasses PTU, and it can add latency because the service first attempts the provisioned path.

The matching Standard deployment should be prepared before the policy is enabled. Microsoft’s setup guidance requires a provisioned deployment and an active pay-as-you-go deployment that matches the model and version in the relevant resource. ^[1] Validate the pairing, API version, authentication path, quota, content-filter behavior, and data-processing boundary in a non-production test. A header that names a nonexistent or unsuitable target is not a capacity plan.

If you want selective control, do not also configure the deployment-level spillover property. The documented precedence rule says that a deployment-level spilloverDeploymentName setting wins when both that property and the request header are present. ^[1] In other words, “per request” should mean per request in both configuration and code.

A practical header policy by workload

The following policy is intentionally conservative. It keeps failure behavior visible instead of silently converting every capacity problem into variable Standard spend.

This table is a policy design, not a Microsoft guarantee. The “fail-fast” decision belongs in your gateway or client before sending the request. For example, a budget-aware gateway can omit the header once the daily Standard-overflow allowance is consumed. It can also omit the header for a model or region that is not on the approved fallback list.

Do not infer spillover from a successful response alone. A Standard response may look identical to a PTU response at the application layer. On spillover, Microsoft documents x-ms-spillover-from-deployment as the originating PTU deployment, x-ms-deployment-name as the deployment that served the request, and x-ms-spillover-error as the original provisioned response code. ^[1] Capture those headers in structured telemetry, subject to your privacy rules. If the Standard attempt also fails, the returned status and body come from that Standard attempt, while the spillover headers remain available to distinguish the path. ^[1]

Monitoring: prove when spillover happened

Use two kinds of evidence. At request level, log a request ID, workload class, header-present flag, originating deployment, serving deployment, original spillover error, final status, and input, cached-input, and output token counts. Never log prompts by default just to prove routing; routing metadata is usually enough.

At resource level, split Azure Monitor metrics by ModelDeploymentName and status code. Microsoft’s guidance also documents an IsSpillover split for identifying records on the Standard deployment that arrived from a provisioned deployment. ^[1] This prevents a common accounting mistake: treating the initial PTU 429 as a failed billable request and then separately counting the successful Standard request as if it were unrelated. The service’s documented metric view places the successful spillover record on the Standard deployment and exposes the spillover dimension there. ^[1]

Build a daily reconciliation with three counts: requests served directly by PTU, requests marked as spillover on Standard, and requests that failed after the spillover attempt. Add token totals to the first two groups. A growing spillover ratio can indicate under-sized PTU, a traffic-shape change, a model-version change, or a Standard quota problem. It does not by itself prove that buying more PTUs is cheaper; compare the incremental PTU hourly cost with the observed Standard token cost and the latency/SLO impact.

Worked cost example

Assume a Global Provisioned deployment runs for a 30-day month. The example uses 730 hours, 30 PTUs, and an illustrative hourly rate of $0.90 per PTU-hour. The rate is a modeling assumption, not a Microsoft quote. Azure says displayed prices are estimates and can vary with agreement, date, region, currency, and offer, so replace it with the current rate for your model and deployment type. ^[5]

Assume 10% of monthly traffic spills to a Standard deployment. The overflow ledger contains 25 million ordinary input tokens, 5 million cached input tokens, and 2 million output tokens. For illustration, use $2.00 per million ordinary input tokens, $0.50 per million cached input tokens, and $8.00 per million output tokens. These token rates are also assumptions; they are not presented as current prices.

The useful conclusion is not the dollar amount. It is the separation of meters. If the overflow share doubles while the PTU count stays fixed, the PTU baseline does not double; the Standard token subtotal changes. If you delete the PTU deployment for part of the month, its hourly line changes, but the fallback token line does not automatically disappear. PTU billing is based on deployed capacity and is prorated for partial-hour existence or resize events; it is not based on the number of tokens consumed. ^[3]

For a more realistic estimate, record the overflow mix by model version and deployment type. Use ordinary input, cached input, output, and any additional model-specific meters separately. Then compare the result with a counterfactual: a larger PTU baseline with less spillover. The decision is a workload and SLO comparison, not a claim that spillover always saves money.

The AI Agent Cost Calculator can help model the monthly baseline and variable usage assumptions. For token-meter detail, see the AI token cost calculator, and use the model pricing comparison when the fallback model or deployment type changes.

Sizing and guardrails before rollout

PTU sizing should use peak request rate, average prompt tokens, average response tokens, cache rate, and model-specific throughput values. Microsoft’s sizing method converts these inputs to normalized TPM and divides by the model’s input TPM per PTU. ^[4] Use that as a starting estimate, then benchmark representative traffic because average token counts can hide burstiness and long-tail prompts.

Set a spillover budget in the same units as the bill. A request count alone is weak because one long completion can cost more than many short requests. A better guardrail has a daily Standard token allowance, a monthly dollar ceiling calculated from current assumed rates, and a maximum spillover percentage by workload. When a threshold is reached, the gateway removes the header for non-essential work or queues it for a later PTU window.

Keep the policy version with every request decision. At minimum, store the workload class, whether the header was present, the named fallback deployment, the PTU attempt status, final serving deployment, final status, and token usage. This makes an invoice reconciliation possible without reconstructing routing from application logs.

Remember that reservations discount the PTU billing meter but do not guarantee model capacity. Microsoft recommends creating deployments and confirming capacity before purchasing a reservation. ^{[2] [3]} A reservation therefore belongs in the fixed baseline line of your model, while spillover belongs in the variable Standard line.

FAQ

Does omitting x-ms-spillover-deployment force a request to Standard?

No. For a request-level policy, omitting the header means spillover is not initiated when the provisioned request returns a non-200 response. The request follows your ordinary error path. That makes header omission the natural fail-fast control, provided a deployment-level spillover property has not already enabled broader behavior. ^[1]

Is spillover charged as an extra surcharge?

The documented cost model has two components rather than a separate spillover surcharge. PTU-served requests use the provisioned hourly cost. Requests routed to Standard use the applicable input, cached-input, and output token rates for that model and deployment type. ^[1] Your estimate should therefore add a PTU capacity line and a Standard token line.

Which response headers show that spillover happened?

Look for x-ms-spillover-from-deployment, which identifies the PTU origin, and x-ms-deployment-name, which identifies the serving deployment. x-ms-spillover-error records the response code that triggered the fallback, such as 429, 400, 500, or 503 in the documented cases. ^[1] Treat these as telemetry fields, not as a substitute for checking the final HTTP status.

Can I use per-request spillover while deployment-level spillover is enabled?

You can send the header, but the documented precedence rule means the deployment-level spilloverDeploymentName setting takes precedence. If the requirement is selective routing, remove the deployment-level setting and manage the header explicitly in the gateway or client. ^[1]

Does spillover guarantee a successful response during a PTU burst?

No. It redirects qualifying overage requests to the Standard deployment, but the Standard attempt can also fail. The final Standard status and body are returned in that case, with spillover evidence still present. Standard capacity, quota, model availability, content policy, authentication, and application timeouts remain relevant failure modes. ^[1]

Should I size PTU for the absolute peak and disable spillover?

Not automatically. Compare a larger fixed PTU baseline with the measured Standard token cost of a smaller PTU deployment, while including latency and reliability requirements. The best choice depends on traffic shape, cache rate, output mix, model, deployment type, and the cost of a failed or delayed request. Use the sizing formulas as an estimate and validate with representative traffic. ^[4]