Self-hosted inference planning
Self-hosted GPU inference cost calculator
Use your cloud quote and benchmarked throughput; the tool does not invent a GPU rental price or promise vendor throughput.
Specialized cost tool
GPU TCO calculator
Translate an editable accelerator rate and measured throughput into monthly spend and token economics.
Scenario output
GPU TCO calculator result
Formula: monthly spend = hourly rate × devices × hours + overhead; capacity = throughput × seconds × devices × utilization.
Utilization and throughput are user benchmark inputs. They are not vendor guarantees, reserved-capacity commitments or performance claims.
Estimate, not a provider quote. Public list rates, regional terms, contract discounts, taxes and usage meters can change. Check the linked source and replace assumptions with your account terms before budgeting.
Pricing and method sources
Rate cards display their own verification and effective dates. Use the original provider source to confirm current billing details.
Use measured throughput, not a GPU marketing number
Tokens per second vary with the model, quantization, prompt length, batch size, sequence length, hardware, runtime, concurrency and quality target. Enter a measurement from your own workload. Utilization is a separate input because an idle GPU still incurs an hourly rental or ownership cost.
Compare the full operating boundary
Include devices, monthly hours, storage, networking, CPU, orchestration, monitoring, engineering and redundancy in your own allowances. Compare the resulting cost per million generated tokens with an API cost estimate only after the workloads, quality and service levels are comparable.
Related: model price comparison and agent monthly-cost planning.
Frequently asked questions
Does the calculator fetch cloud GPU rates?
No. Rental rates depend on provider, region, machine shape, reservation, spot status and availability. Enter a current quote or invoice rate.
What throughput should I use?
Use a benchmark for the same model, quantization, prompt/output lengths, batching, runtime and concurrency. The output is only as reliable as this workload measurement.
Does utilization reduce the rental fee?
No. Utilization affects the amount of work served per paid hour, not the billed GPU-hours in a rental scenario.
Can I include engineering or redundancy?
Yes. Enter those costs in monthly overhead or include additional devices and reserve capacity explicitly.