Research-backed calculator guide

GPU Capacity Reservations vs On-Demand: Pay for Assurance or Flexibility?

Compare GPU capacity reservations, capacity blocks, commitments, and on-demand pricing with a utilization break-even example for AI training and inference.

Reservation, commitment, capacity block, and on-demand are different

A capacity reservation protects access to a defined resource. The definition usually includes the GPU or VM family, quantity, region, availability zone, and sometimes placement or tenancy. AWS describes On-Demand Capacity Reservations as zone-specific capacity assurance, while Capacity Blocks for ML reserve GPU-based instances for a future date and a defined period.^{[1] [2]}

A usage commitment or reserved-instance-style product primarily changes the billing rate for eligible usage. It may require a term, a spending commitment, or a matching configuration. It does not automatically solve a regional capacity shortage. Azure’s documentation makes this distinction explicit: capacity reservations secure deployment availability, while Reserved Instances provide a billing discount and do not guarantee capacity.^[3]

A capacity block is a time-bounded reservation. It can fit a known training or fine-tuning run, a scheduled benchmark, or a temporary inference surge. AWS says its Capacity Blocks for ML can be scheduled up to eight weeks ahead, with up to 64 instances in a block and up to 256 instances across blocks, subject to supported types and regions.^[2] Those limits are product rules, not universal GPU-market assumptions.

On-demand means you request eligible capacity when you need it and pay for the runtime or usage meter. That gives you flexibility to stop early, change the GPU shape, or postpone the job. It also leaves you exposed to quotas, regional supply, provisioning delays, and the possibility that a large matching cluster is unavailable at the required time.

The words matter because the wrong comparison can produce a false saving. Comparing a one-year usage discount with a one-week capacity block mixes rate certainty, availability assurance, and time horizon in one number.

What should GPU capacity reservation cost include?

Start with the GPU or VM charge, then add every cost that changes between the options. For a fixed block, include all reserved GPU-hours even if the job is idle. Add CPU, memory, boot volume, checkpoint storage, data transfer, orchestration, support, and any warm-up or staging environment. For on-demand, include the same components plus expected retry, queue, delay, and partial-cluster costs.

Also price the cost of delay. A training run that starts 18 hours late may miss an evaluation window, a customer demo, or a downstream launch. An interactive inference service may lose requests or violate an SLO while its replacement GPUs are being found. These are business assumptions, not vendor quotes, so record them separately from the infrastructure bill.

The vendor pages illustrate why quote context is necessary. Runpod’s pricing page, updated October 28, 2026, lists an H100 SXM at $3.99 per hour in its displayed Pods table and directs buyers seeking reserved capacity and contract pricing to enterprise sales.^[4] Lambda shows H100 instance prices by configuration and separate multi-GPU cluster price bands, with duration and GPU count affecting the displayed GPU-hour rate.^[5] These observations are dated, provider-specific examples, not a market average or an apples-to-apples benchmark. Confirm the exact GPU variant, host configuration, region, billing unit, taxes, and contract terms before using a vendor number in a business case.

Worked cost example

Consider a team that needs eight identical GPUs for a deadline-bound fine-tuning run. The team is comparing an on-demand request with a hypothetical fixed 168-hour capacity block. The following rates are assumptions for the calculation, not a quoted price:

• On-demand rate: $4.00 per GPU-hour.
• Capacity-block rate: $3.40 per reserved GPU-hour.
• Reserved quantity: 8 GPUs.
• Reservation window: 168 hours (seven days).
• Expected productive use: 120 hours of the window.
• Exclusions: taxes, storage, network transfer, host CPU, orchestration, support, and the value of a missed deadline.

At the assumed rates, the block is cheaper only when the team productively consumes more than 85% of the reserved GPU-hours. The 120-hour plan consumes 120 ÷ 168 = 71.4%, so on-demand wins on raw GPU cost by $729.60. If the team uses the full 168 hours, on-demand would cost 1,344 × $4.00 = $5,376, and the block would save $806.40.

Now add schedule risk. Suppose an on-demand launch has a 20% chance of delaying the run by 18 hours, and the team values that delay at $6,000 because of downstream engineering and launch consequences. The expected delay cost is 20% × $6,000 = $1,200. Under this assumption, the on-demand expected cost becomes $3,840 + $1,200 = $5,040, above the block’s $4,569.60 raw cost. That does not prove the block is the right purchase; it shows how the answer changes when availability risk is made explicit.

The calculation has two boundaries. First, a provider may price capacity assurance above, below, or equal to ordinary on-demand usage. Azure documents capacity reservations at the underlying VM rate, with unused reserved units still billed, so “reservation” should not be treated as an automatic discount.^[3] Second, a capacity block may have targeting, cancellation, termination, or supported-region rules. AWS, for example, says Capacity Blocks must be targeted by reservation ID, cannot be canceled, and terminate on a documented schedule.^[2]

A practical decision test

Choose on-demand while the workload is changing

Use on-demand for prototypes, new model serving, early traffic, irregular notebooks, and experiments where the GPU model or count may change. It gives the team a cheap way to measure actual throughput, model-loading time, memory pressure, and cost per completed job before locking in a shape.

It is also useful above a proven baseline. If a production service normally needs four GPUs but sometimes needs twelve, a four-GPU baseline can be covered by a suitable commitment or reservation while the extra eight are requested on-demand. This avoids paying for the forecast peak every hour. The approach still needs quotas, alternate regions, queue limits, and a tested degradation path.

Reserve when the start window is financially important

Reserve capacity when a specific date, cluster size, or zone matters more than the option to release resources early. Examples include a distributed training run with a fixed evaluation date, a customer launch with a contractual milestone, or a recovery environment that must be able to start during a regional incident.

Before you reserve H100 capacity or another scarce GPU type, verify that code, containers, drivers, credentials, quotas, data, networking, and checkpoint export are ready. A reservation cannot recover time lost to an unready image or a data pipeline that feeds the GPUs too slowly. A smaller preflight run can test that the workload is genuinely ready for the fixed window.

Buy a rate commitment only after measuring the baseline

A usage discount is suitable when the compatible workload is stable enough to consume the commitment. Measure utilization by the matching account, GPU family, region, zone, host shape, and service scope. Aggregate company-wide demand can look high while the eligible pool remains underused.

Use the simple raw break-even rule:

break-even utilization = effective committed rate ÷ comparable on-demand rate

If the commitment costs $3.40 per covered GPU-hour and comparable on-demand costs $4.00, the break-even point is 85%. Below that, paying for unused commitment is more expensive on raw compute. Above it, the discount can help, provided the workload is eligible and the capacity itself is available.

What a procurement quote must answer

Ask the provider to state whether the offer supplies a rate discount, a capacity guarantee, or both. Then ask for the exact GPU model, memory variant, number of GPUs, region, availability zone, host topology, start and end time, minimum duration, billing increment, cancellation policy, and treatment of unused capacity.

For a future block, ask when billing begins, whether the quoted total is fixed, what happens if the job ends early, and how close to the end time instances are terminated. For a long-term commitment, ask which workloads qualify and what happens if the model, region, account, or architecture changes. For on-demand, ask about quota approval, typical provisioning process, alternate regions, and whether a smaller or different GPU can be substituted.

Do not compare a single GPU-hour line item when the real requirement is an eight-GPU synchronized cluster. Network topology, host memory, storage throughput, and placement can determine whether the job completes at all. The right unit may be cost per completed training run, cost per million generated tokens, or cost per successful inference request, not cost per GPU-hour.

For a broader infrastructure estimate, use the self-hosted GPU inference cost calculator and carry the resulting GPU, storage, and utilization assumptions into the reservation worksheet. If the workload is an API-mediated design rather than self-hosted inference, compare the model meters in the model pricing comparison. Retrieval-heavy services should also include the RAG, embedding and vector database cost calculator, because reserving GPUs does not remove embedding, vector, or storage spend.

Limitations and a safer operating pattern

Prices, supported GPU types, regions, and reservation mechanics change. The dated observations above should be refreshed before a purchase. A provider page may show a public on-demand rate while keeping reserved-capacity pricing behind a sales process. Estimates also omit taxes and contractual terms unless explicitly included.

A blended portfolio is often safer than an all-or-nothing choice: cover a measured baseline, keep on-demand for bursts, and use interruptible or spot capacity only for checkpointable work. Put a stop timer on on-demand resources, export checkpoints before a fixed block ends, and monitor covered utilization. Revisit the mix when traffic, model size, or deployment region changes.

The decision rule is simple: buy the least commitment that satisfies the availability requirement. If an idle reservation is more expensive than the expected delay risk, stay flexible. If a missed start would be more expensive than unused reserved hours, pay for assurance—but only after verifying that the workload and provider terms match.

FAQ

Is a GPU capacity reservation automatically cheaper than on-demand?

No. A reservation may secure capacity at the ordinary underlying rate, at a premium, or at a discounted fixed-window rate. The answer depends on the quoted rate, the hours you will actually use, unused-capacity billing, and the value of avoiding a delayed launch. Calculate cost per completed job rather than assuming that the word “reserved” means “discounted.”

What is the difference between a GPU capacity block and on-demand?

A capacity block reserves a defined GPU quantity for a specified future window. It trades flexibility for schedule assurance and can charge for the full window even if the job ends early. On-demand lets you request and release capacity as needed, but the required GPU type and quantity may not be available at the desired time. Product rules vary by provider, so check start, end, targeting, and cancellation terms.

Should I reserve H100 capacity for production inference?

Reserve it when the service has a stable minimum footprint and a capacity shortfall would violate a meaningful availability or latency objective. Do not reserve forecast peak capacity automatically. Cover the measured baseline, keep a tested burst path, and verify that the reservation matches the H100 variant, region, zone, host topology, and orchestration layer your service actually uses.

How do I calculate guaranteed GPU availability pricing?

Separate the infrastructure quote from the value of assurance. First calculate reserved GPU-hours × reserved rate, then add fixed platform, storage, transfer, and support costs. Separately estimate the probability and financial impact of a late or failed on-demand launch. Compare the two totals under explicit assumptions. Do not treat a public on-demand price as proof of guaranteed availability.

What utilization makes a GPU commitment worthwhile?

For a like-for-like fully paid commitment, divide the effective committed rate by the comparable on-demand rate. A $3.40 committed rate against a $4.00 on-demand rate breaks even at 85% eligible utilization. That is only a raw-compute threshold; scope mismatch, migration risk, idle replicas, taxes, financing, and engineering effort can move the real threshold higher.