Research-backed calculator guide
Azure OpenAI PTU Sizing from P50, P95 and P99 Traffic
A practical method for Azure OpenAI PTU sizing by percentile: convert P50, P95 and P99 traffic into normalized TPM, round for deployment increments, and compare the capacity and cost trade-offs.
Quick answer
Azure OpenAI PTU sizing by percentile means sizing a deployment from the distribution of your traffic, rather than from one average or the absolute maximum. Build one-minute buckets from representative production or load-test data. For each bucket, calculate normalized tokens per minute (TPM):
normalized TPM = input TPM × (1 − cache rate) + output-to-input ratio × output TPM
Then take the P50, P95 and P99 of those normalized-TPM buckets. For each percentile, divide by the selected model’s Input TPM per PTU, then round up to the deployment’s PTU increment and minimum. Microsoft’s sizing guide documents this normalized-TPM method and cautions that the final estimate should be checked against representative traffic rather than treated as a substitute for benchmarking.[1]
The decision is a service objective, not a mathematical race to the largest number. P50 describes a typical minute and usually leaves bursts outside the provisioned deployment. P95 covers more of the observed distribution. P99 provides more headroom, but it can reserve capacity for rare, narrow spikes that may be cheaper to handle with an approved spillover path or another operational control. The right target is the highest percentile your latency, error-rate and budget requirements justify.
PTUs are dedicated processing capacity, and Microsoft says they are billed hourly based on the deployed amount rather than the number of tokens consumed. Throughput still varies by model, deployment type, request shape and cache behavior, so do not reuse one model’s PTU ratio for another.[2]
Scope boundary: this is a percentile-to-PTU capacity worksheet, not a decision guide for whether PTU is economically preferable to PAYG or reservations. For the broader PTU-versus-PAYG break-even decision, see the Azure OpenAI PTU vs PAYG guide.
What percentile sizing actually measures
A traffic percentile is not a latency percentile. P95 traffic answers, “What normalized TPM was at or below this value in 95% of the observed one-minute buckets?” It does not promise that 95% of requests will meet a latency target. Latency, throttling, retries and queueing need separate measurements.
P50, P95 and P99 are useful because a PTU deployment is a fixed-capacity commitment. The median can describe the center of the workload, while P95 and P99 show progressively more burst protection. If the distribution is flat, the three answers may be close. If traffic has short, tall bursts, P99 can be much larger than P95 even though the spike occupies only a small part of the observation window.
For each one-minute bucket t, retain the inputs that drive capacity:
RPM_t: completed or attempted calls in the bucket, using one consistent definition.input_t: input tokens in the bucket.output_t: generated output tokens in the bucket.cache_rate_t: the fraction of input tokens served from prompt cache, when the selected model’s accounting supports that treatment.r: the model-specific output-to-input ratio.
If token totals are already measured, you do not need to multiply RPM by an average request size. If only request-level logs are available, aggregate prompt and output tokens first. For a bucket with 100 calls, 500 input tokens per call and 100 output tokens per call, input TPM is 50,000 and output TPM is 10,000.
The important analytical choice is to calculate a normalized TPM value for every bucket and then take the percentile of that composite series. Taking P95 RPM, P95 input size and P95 output size from three different distributions can produce a deliberately conservative number, but it can also describe a combination that never occurred. If you use that shortcut, label it as a planning upper bound. A bucket-level composite percentile is usually easier to explain and audit.
The Azure OpenAI PTU sizing formula
The official formula has four steps:
input TPM = RPM × average prompt tokensoutput TPM = RPM × average response tokensnormalized TPM = input TPM × (1 − cache rate) + output-to-input ratio × output TPMraw PTUs = normalized TPM ÷ Input TPM per PTU
The model and deployment type determine the Input TPM per PTU, output-to-input ratio, minimum deployment and scale increment. Microsoft’s current sizing table is therefore the authority for the selected model and deployment, not a generic PTU rule.[1]
For example, the current table observed on 2026-11-02 lists GPT-5.2 with 3,400 input TPM per PTU and an output-to-input ratio of 8 for the illustrated provisioned deployment types. Those values are product parameters, not a universal conversion factor. Confirm them again when your model version or deployment type changes.
Cache rate deserves special care. If 20% of input tokens are cache-served, the formula uses 80% of input TPM for the relevant accounting model. Output tokens remain in the output term. A cache hit rate guessed from a dashboard headline is not enough if cache behavior changes by route, tenant, prompt prefix or time of day; compute it for the same buckets used for the percentile.
After the raw division, round up. A raw result of 76.24 PTUs is not a deployable 76.24 PTUs. Use the greater of the model/deployment minimum and the next supported scale increment. The increment may differ between Global Provisioned, Data Zone Provisioned and Regional Provisioned, and the minimum can be materially different as well.[1]
Worked cost example: P50 versus P95 versus P99
The table below is an original planning example, not a vendor quote. It assumes GPT-5.2, a ratio of 8, 3,400 input TPM per PTU, a 15-PTU minimum and a 5-PTU increment. The prompt and response values are averages within each percentile slice. The calculation rounds up to the next five PTUs.
| Traffic slice | RPM | Input tokens/request | Output tokens/request | Cache rate | Input TPM | Output TPM | Normalized TPM | Raw PTUs | Rounded PTUs |
|---|---|---|---|---|---|---|---|---|---|
| P50 | 100 | 500 | 100 | 25% | 50,000 | 10,000 | 117,500 | 34.56 | 35 |
| P95 | 180 | 600 | 120 | 20% | 108,000 | 21,600 | 259,200 | 76.24 | 80 |
| P99 | 250 | 700 | 140 | 15% | 175,000 | 35,000 | 428,750 | 126.10 | 130 |
The P50 row is calculated as (50,000 × 0.75) + (8 × 10,000) = 117,500 normalized TPM; 117,500 ÷ 3,400 = 34.56, which rounds to 35 PTUs. The P95 row becomes (108,000 × 0.80) + (8 × 21,600) = 259,200; the raw requirement is 76.24, so the deployment rounds to 80 PTUs. The P99 row becomes (175,000 × 0.85) + (8 × 35,000) = 428,750; 126.10 raw PTUs becomes 130.
To make the cost denominator visible, suppose the Azure pricing page’s GPT-5.2 Global provisioned listing observed on 2026-11-02 is used as an illustrative vendor-listed rate of $1 per PTU-hour. Azure states that its displayed prices are estimates and can vary by agreement, date, currency and offer, so this is not a negotiated quote or a promise of your invoice.[4] Using 730 hours per month:
- P50:
35 PTUs × $1/PTU-hour × 730 hours = $25,550/month. - P95:
80 PTUs × $1/PTU-hour × 730 hours = $58,400/month. - P99:
130 PTUs × $1/PTU-hour × 730 hours = $94,900/month.
Under those assumptions, selecting P99 instead of P95 adds 50 PTUs and $36,500/month of provisioned hourly capacity. The difference is not a claimed saving or a forecast. It is the arithmetic consequence of the stated rate, 730-hour convention and rounded PTU counts. It excludes taxes, support, reservation terms, standard-deployment spillover tokens, discounts, exchange rates and any negotiated commercial agreement.
For a real estimate, place the verified rate for the exact model, region/deployment type and billing mode beside the PTU calculation. The Azure OpenAI cost and PTU calculator can hold the financial assumptions, while the percentile worksheet should preserve the raw traffic evidence. For broader comparisons, use the model pricing comparison rather than mixing a PTU hourly denominator with token prices from a different deployment.
How to choose P50, P95 or P99
Use P50 as a baseline diagnostic, not as a promise of production coverage. It is useful for identifying whether the deployment would be mostly idle or whether a workload is consistently large enough to justify dedicated capacity. It is rarely sufficient for an interactive service that must absorb ordinary bursts without routing or queueing controls.
P95 is a practical starting point when the service can tolerate a defined tail policy. Document what happens to the remaining five percent of bucket demand: queue it, shed it, retry with bounded backoff, route it to a standard deployment, or accept a lower service tier. That policy is part of the capacity design. It should not be hidden inside a rounded PTU number.
P99 is appropriate when the tail has a high business cost, such as a strict interactive SLO, a batch deadline with little recovery time, or a regulated workflow where fallback is not acceptable. It can also be wasteful when the P99 is caused by a few short and predictable events. Compare the incremental PTU-hour cost with the operational cost of handling those events. Microsoft’s right-sizing discussion explicitly describes peak provisioning and a baseline-plus-spillover pattern as different trade-offs rather than one universally correct answer.[3]
Do not use the maximum automatically. A single deployment incident, replay storm or test run can dominate the maximum while having no bearing on normal capacity. Keep the maximum as a stress-test case and investigate it separately. If the maximum is a known launch event, model that event as a temporary scenario with its own duration and routing plan.
Request shape, quota and capacity checks
Average tokens can hide a serious shape problem. Two workloads can have the same average TPM while one consists of many small calls and the other contains fewer long prompts and large generations. Microsoft notes that request/call shape affects capacity for older models, and its current guidance still recommends benchmarking representative traffic for the most accurate result.[1] Keep prompt length, completion length, cache rate and concurrency distributions in the evidence set instead of storing only one monthly token total.
A PTU estimate also does not reserve Azure capacity by itself. Quota is a policy limit, while capacity is the deployable amount available for a model version in a region at that time. Microsoft says quota and capacity are distinct and that having quota does not guarantee that deployment capacity is available.[2] Before treating the P95 or P99 result as purchasable, verify the selected deployment type, region, model version, minimum, increment, quota and live capacity in the Foundry experience.
Keep the sizing worksheet versioned. Record the model name and version, deployment type, region or data zone, observation window, bucket width, percentile method, token meter definitions, cache-rate method, ratio, TPM-per-PTU value, minimum, increment, hourly or reservation assumption and the date checked. That turns a calculator result into a reproducible capacity decision.
When spillover changes the decision
If you choose less than P99, define the overflow path before deployment. Microsoft documents spillover from a provisioned deployment to a corresponding standard deployment when the provisioned deployment returns eligible non-200 responses, including a 429 when PTUs are fully used.[5] The standard deployment’s input, cached-input and output tokens are billed at its applicable rates, while provisioned requests incur the PTU hourly cost.
Spillover is not a free extension of the PTU count. It changes the denominator from “all traffic is covered by PTUs” to “base PTUs plus an estimate for overflow tokens.” Measure the IsSpillover traffic and token totals after launch, compare them with the assumed tail, and update the percentile decision. If overflow is unacceptable, size closer to P99 or redesign the workload rather than silently treating PAYG fallback as guaranteed capacity.
Run your scenario
Turn this framework into an editable estimate.
Use dated, source-linked assumptions and see cost per request, successful task and month.
FAQ
Is P95 always the right Azure OpenAI PTU target?
No. P95 is a decision point, not a platform default. Choose it when the remaining tail has a documented queue, retry, spillover or service-tier policy. Choose P99 when the cost of tail degradation is higher than the incremental PTU commitment. Compare both with the same model parameters and cost denominator.
Should I calculate P95 separately for RPM, input tokens and output tokens?
Prefer a bucket-level normalized-TPM series when the data is available. It preserves the relationship between calls and token shape and avoids combining three different high-percentile moments that may not occur together. If you only have summary percentiles, calculate the conservative composite explicitly and label it as an upper-bound planning estimate.
Does prompt caching reduce the PTUs I need?
It can reduce the input portion of normalized TPM when the selected model’s PTU accounting treats cached input as capacity-exempt or otherwise discounts it as documented. Use an observed cache rate by bucket, and recheck the model-specific guidance. Caching does not remove output-token capacity, and a cache rate from one route should not be applied to every workload.
Are PTUs interchangeable across Azure OpenAI models?
The quota concept can be model-independent, but throughput per PTU varies by model and deployment type. The same PTU count therefore does not imply the same TPM for GPT-5.2, GPT-4.1, an image model or another offering. Retrieve the current model-specific TPM-per-PTU, ratio, minimum and increment before calculating.
What should I do when P99 is much more expensive than P95?
First confirm that the gap is real and not caused by a logging artifact, a short test, retries counted as new demand or a mixed workload. Then price the P95 base plus a measured overflow policy, and compare it with P99 using the same hours and rates. If you use spillover, validate the matching standard deployment and monitor spilled requests; do not count an unmeasured fallback as guaranteed service.
How often should I recalculate PTU capacity?
Recalculate when the model or version, deployment type, region, prompt template, output limit, cache behavior, traffic mix or service objective changes. Even without a configuration change, a new representative observation window can reveal drift. Keep the previous percentile series so that a larger PTU number is explainable rather than appearing as an unexplained budget increase.
For token-meter definitions and a separate input/output cost view, the AI token cost calculator is a useful companion. For a full workload-level monthly estimate that includes other services, compare it with the LLM cost calculator. Neither replaces the model-specific PTU parameters or a representative capacity test.
Primary sources checked
Pricing and product behavior can change. These sources support the time-sensitive claims in this guide; verify current provider terms before committing spend.