Research-backed calculator guide
Azure Batch Enqueued-Token Quota Planner
Plan Azure OpenAI Batch admission with enqueued-token headroom, file limits, shard sizing, and a dated cost example.
Quick answer
An Azure OpenAI Batch quota planner should answer two separate questions: can this job be admitted without overrunning the enqueued-token limit, and what will the token work cost if it is admitted? Microsoft describes Global Batch quota in enqueued tokens. The tokens in a submitted file count against the total until the job reaches a terminal state, so a file that is small by itself can still be rejected when other jobs are active.[1] The Batch file limits are separate constraints: Microsoft currently documents 500 input files without expiration, 10,000 when expiration is set, 200 MB per input file, 1 GB when using bring-your-own storage, and 100,000 requests per file.[1]
Use this planning proxy before submission:
planned queue = current active enqueued tokens + proposed job estimate × (1 + safety buffer)
Admit the job only when planned queue is below the quota configured for the model and subscription tier. Treat the proposed job estimate as an engineering estimate, not as a promise about Azure's internal tokenizer or admission algorithm. For a safer first pass, estimate input tokens from the actual JSONL payload, add a documented output reserve if your request body sets one, then hold back 10% to 20% of quota for estimation error and small follow-up jobs.
Batch is a deferred-workload choice, not a real-time concurrency switch. Microsoft says it targets completion within 24 hours and prices eligible Batch work at 50% less than Global Standard; completed work remains billable if a job is cancelled.[2] The pricing page says displayed prices are estimates, not customer-specific quotes, and rates vary by agreement, date, currency, model, and deployment type.[3] Therefore, use the worksheet for admission and scenario planning, then verify the current portal quota and price before committing production work.
What the planner measures
1. The quota ceiling
Start with the quota that applies to the target Batch deployment and subscription. The Microsoft quota table separates Global Batch from Data Zone Batch and shows different limits by model and subscription category.[1] For example, on the page checked on November 2, 2026, the listed default Global Batch quota is 200 million enqueued tokens for GPT-5 and 1 billion for GPT-5 mini; the same page lists different Enterprise and MCA-E values. Those are examples from the current table, not a universal entitlement. Record the exact model, deployment type, subscription tier, and region in your own worksheet.
Do not substitute ordinary online TPM for Batch enqueued-token quota. A Microsoft Q&A case shows a deployment configured with a 30,000-token queue limit rejecting a new job even though each individual JSONL file was under 30,000 tokens. The accepted explanation was that the combined active jobs, rather than one file, exhausted the queue.[4] That historical case is useful for understanding the failure mode, but its number should not be reused as your current quota.
2. The active queue
current_active_tokens is the sum of the token estimates for jobs that have been submitted but have not reached a terminal state. Keep a row per job with its submission time, deployment, estimated input tokens, output reserve, status, and last observation. When a job completes, fails, or is cancelled, remove it from the active headroom calculation according to your operational definition of terminal state.
The value is not a billing total. It is a capacity-control estimate. Keep actual usage and invoice reconciliation separate. The AI token cost calculator is useful for normalizing input, output, and cached-token assumptions, while this planner is focused on whether a Batch submission fits the queue.
3. The proposed job estimate
The official Batch guide requires JSON Lines input, a custom_id for mapping responses, and a consistent model deployment name on every line. Microsoft also recommends submitting large files rather than many small files with only a few lines.[2] That guidance gives you a practical measurement boundary: estimate the request payloads you will actually upload, not a vague monthly token target.
A simple internal estimator can store these fields for each line:
input_tokens_estimate: token count for the system, user, and other input content in the request.output_reserve: the maximum or expected output allowance used by your application for planning.request_estimate: input plus the output reserve, documented as a proxy.custom_id: the stable key used to reconcile results because responses are not returned in input order.[2]
If your request mix contains short classifications and long document summaries, do not use one average without checking the tail. Calculate at least P50 and P95 line estimates. A P95-based shard prevents a few large requests from making an apparently safe average misleading.
4. File and request constraints
Admission can fail even when token headroom exists. Check the number of input files, bytes per file, and requests per file against the current quota page. Set expiration when appropriate because Microsoft says it raises the per-resource input-file limit from 500 to 10,000; expiration does not change the token quota.[1] If the payload is larger than the direct-upload limit, Microsoft documents Batch with Azure Blob Storage and a 1 GB BYOS input-file limit.[1]
A shard is valid only if it passes all constraints. In practice, choose the smallest of three ceilings: remaining safe tokens, the request-count ceiling, and the byte ceiling. A shard that fits tokens but exceeds 100,000 requests per file is not admissible. A shard that fits requests but creates too many files increases file-management pressure and may make retries harder to reconcile.
Worked quota-planning example
Assume a Global Batch GPT-5 deployment. The worksheet uses a 200,000,000-token default quota only as an example taken from the Microsoft table checked on November 2, 2026.[1] Your portal may show another value. Assume the active queue contains two nonterminal jobs estimated at 42,000,000 and 36,000,000 tokens. A new document-classification workload contains 96,000,000 estimated tokens. Use a 15% planning buffer, and cap a single planned shard at 80% of quota so one submission does not consume the entire queue.
| Step | Input or assumption | Calculation | Output |
|---|---|---|---|
| Quota ceiling | 200,000,000 tokens | Vendor-table example | 200,000,000 |
| Active job A | 42,000,000 tokens | Recorded estimate | 42,000,000 |
| Active job B | 36,000,000 tokens | Recorded estimate | 36,000,000 |
| Current active queue | — | 42,000,000 + 36,000,000 | 78,000,000 |
| Proposed workload | 96,000,000 tokens | Sum of line estimates | 96,000,000 |
| Safety buffer | 15% | 96,000,000 × 0.15 | 14,400,000 |
| Planned proposed queue | — | 96,000,000 + 14,400,000 | 110,400,000 |
| Queue after submission | — | 78,000,000 + 110,400,000 | 188,400,000 |
| Remaining headroom | — | 200,000,000 − 188,400,000 | 11,600,000 |
| Single-shard guardrail | 80% of quota | 200,000,000 × 0.80 | 160,000,000 |
The submission passes this proxy because 188.4 million is below the 200 million ceiling, and the proposed workload plus buffer is below the 160 million single-shard guardrail. It leaves only 11.6 million tokens of estimated headroom, so the scheduler should pause additional large submissions until one of the active jobs reaches a terminal state. If the same workload were submitted while the active queue was 95 million, the projected total would be 205.4 million and the planner would reject it or split it into smaller shards.
Suppose each file is capped at 100,000 requests and the 96 million-token workload has 240,000 requests. The request constraint requires at least three files even though the token calculation might fit one. If the 96 million tokens are split evenly into three files, each contains about 32 million tokens before the buffer. If a byte check shows that one file would exceed 200 MB, create more files or use the documented Blob Storage path. Keep every line's deployment name identical within a file, and preserve custom_id values across retries.[2]
This example intentionally does not claim that the 15% buffer or 80% guardrail is an Azure rule. They are changeable planning assumptions. Tune them from observed estimation error, job duration, cancellation behavior, and the cost of a rejected submission. The Microsoft Q&A guidance also recommends tracking active jobs and controlling submission with queuing or throttling when the queue is crowded.[4]
Worked cost example
Quota fit and price are related but not interchangeable. For a dated illustration, use the Azure pricing page's listed GPT-5 Global rates checked on November 2, 2026: Standard input $1.25 per 1 million tokens and output $10 per 1 million tokens; Batch input $0.63 per 1 million and output $5 per 1 million.[3] These are vendor-listed prices in USD per million tokens, not a quote. They exclude any separate services, taxes, negotiated terms, or pricing changes after the check date.
Assume the accepted workload consumes 120 million input tokens and 30 million output tokens:
- Batch input:
120 × $0.63 = $75.60. - Batch output:
30 × $5.00 = $150.00. - Estimated Batch model-token total: $225.60.
- Standard input:
120 × $1.25 = $150.00. - Standard output:
30 × $10.00 = $300.00. - Estimated Standard model-token total: $450.00.
- Illustrative difference: $224.40, or 50% lower under these listed rates.
The result is a model-token estimate, not a complete Azure invoice. It assumes all 120 million input and 30 million output tokens use the listed GPT-5 Global meters, and it does not include storage, monitoring, application compute, retries, failed work, or other Azure services. The Batch guide says cancellation returns completed work but charges for work already completed, so a cancellation plan should include that exposure.[2] If your workload uses another model, Data Zone, a different subscription offer, or a newer pricing table, replace the rates rather than carrying this example forward. For a broader comparison across models and deployment choices, use the Azure OpenAI cost and PTU calculator and compare assumptions in the model pricing comparison.
Admission checklist
Before uploading a file, record the quota table row and deployment identity. Estimate input tokens from the actual JSONL payload, add the output reserve used by your service, and write down the buffer percentage. Add the estimates for every nonterminal job targeting the same quota. Reject or shard the new work if the projected total exceeds quota or your internal guardrail.
Then check non-token limits: input-file count, expiration policy, file bytes, requests per file, storage path, and consistent deployment names. Prefer a few meaningful large files over many tiny files, but stop at the first binding constraint. Generate stable custom_id values and retain a manifest that maps each shard to its source rows. After submission, monitor job status and recompute headroom when a job reaches a terminal state.
If the job must finish interactively, do not force Batch simply because its token price is lower. Microsoft positions Batch for asynchronous work with a 24-hour target rather than an interactive latency guarantee.[2] A queue, throttled Standard deployment, or another capacity design may be more suitable. The LLM cost calculator can help compare the recurring token scenario, while this page's worksheet keeps the admission decision explicit.
Run your scenario
Turn this framework into an editable estimate.
Use dated, source-linked assumptions and see cost per request, successful task and month.
FAQ
Does Azure OpenAI Batch quota equal tokens per minute?
No. Global Batch quota is described in enqueued tokens, while ordinary online deployments expose other throughput controls. The quota page says tokens in a submitted file count until the job reaches a terminal state.[1] Treat the two meters as separate fields in a capacity model. Do not infer Batch admission from the online TPM value shown for another deployment.
Can a small JSONL file still trigger token_limit_exceeded?
Yes. A small file can be rejected when the combined estimate of active and newly submitted work exceeds the configured queue limit. The Microsoft Q&A example shows exactly that pattern: each file was under 30,000 tokens, but the deployment's combined queue was already full.[4] Track all nonterminal jobs for the target quota, not just the next file's size.
How many requests can one Batch input file contain?
The Microsoft quota page currently lists a maximum of 100,000 requests per Batch input file.[1] That is independent of the token and byte checks. If a workload needs 240,000 requests, a planner must create at least three files before considering token headroom. Recheck the current page because service limits can change.
Should I use a 15% safety buffer?
Fifteen percent is a transparent example, not a Microsoft requirement. Choose a buffer from your token-estimation error and the impact of a rejected or delayed job. If requests vary widely, use a P95 or P99 estimate and a larger buffer. Keep the percentage in configuration so a scheduler can explain why it accepted or rejected a submission.
Does Batch always cost exactly 50% less?
No. Microsoft's Batch guide and pricing page describe a 50% discount relationship for eligible Global Standard pricing, but the actual amount depends on model, deployment, availability, pricing date, offer, and what else appears on the invoice.[2] [3] The worked example is dated and assumes specific GPT-5 Global meters. Replace those rates with the current official table before budgeting.
What happens if a Batch job runs longer than 24 hours?
The documented 24 hours is a target, not an automatic expiry. Microsoft says the service does not expire a job merely because it takes longer; you can cancel it, and completed work is returned while completed work remains billable.[2] Include that operational risk in any deadline decision rather than treating the target as an SLA.
Primary sources checked
Pricing and product behavior can change. These sources support the time-sensitive claims in this guide; verify current provider terms before committing spend.