Research-backed calculator guide

Vertex AI Endpoint Autoscaling Cost: Minimum Replicas, Cold Starts and Throughput

Learn how to estimate Vertex AI endpoint autoscaling cost from node-hours, minimum replicas, peak throughput, startup lag, and scale-to-zero eligibility.

What you are actually paying for

An endpoint is a serving resource, not merely a URL. Deploying a model associates physical compute resources with it so the model can answer online inference requests. Google’s deployment guide says the compute settings are part of the deployment decision, and changing them generally requires a redeployment. ^[4] This is why an autoscaling estimate should start with the deployed model’s machine type and accelerator configuration rather than with request count alone.

Google’s compute guidance describes inference nodes as virtual machines allocated for online or batch inference. Machine types differ in vCPU, memory, and price. GPUs are optional for some workloads, but when attached they change the cost and the amount of work a replica can perform. The same guidance points to autoscaling, reservations, and Spot VMs as separate levers for availability and cost management. ^[3]

There are four distinct inputs in a practical Vertex AI endpoint cost model:

1. The rate: use the price for the actual region, machine type, prediction mode, accelerator, and billing or commitment option. This is the vendor-quoted input, not an estimate.
2. The replica schedule: estimate how many nodes are active in each traffic window. A minimum of one means one node continues running even when requests are quiet, subject to the product’s actual behavior.
3. The scale lag: include warm capacity when a burst must be served immediately. Nodes that arrive after the burst may lower the eventual average cost but still miss the latency objective.
4. The scope: keep endpoint serving cost separate from token-based generative model charges, data storage, Cloud Logging, egress, evaluation, and downstream services.

The pricing page also makes an important distinction between online and batch prediction. Online autoscaling prioritizes the latency of individual requests and can keep a model ready for a few idle minutes after a request. More frequent requests can therefore increase the number of nodes and the monthly charge. ^[1] A quiet endpoint can still cost money when its minimum replicas remain ready.

How minimum replicas change the estimate

minReplicaCount is not a monthly price by itself. It is a floor on capacity. To price that floor, multiply the minimum by the number of hours the endpoint is deployed, then add incremental node-hours from higher traffic windows.

For a rough calendar month, you can use 744 hours (31 × 24) as an assumption. A 30-day month uses 720 hours. State which one you use because a 24-hour difference is meaningful for a high-rate GPU replica. The safer production method is to export actual node-running seconds from billing or monitoring and reconcile that against the estimate.

A minimum of one has two effects. It creates a predictable warm baseline and reduces the chance that every quiet-period request waits for a new VM, container, and model load. It also creates a recurring charge even if the endpoint receives no requests. A minimum of three gives more failover and burst headroom, but its quiet-period cost is approximately three times the one-replica floor before any scaling above three.

The best minimum is therefore not “the lowest number that saves money.” It is the smallest warm capacity that passes a measured latency and availability test for the baseline traffic shape. Google recommends monitoring CPU or accelerator usage, requests, latency, and current versus target replica counts. It also cautions that a replica runs a single container, so a model that cannot use the selected machine effectively may not cause the autoscaler to add nodes as expected. ^[2]

Cold starts and throughput are part of cost

A cost-only comparison can make scale-to-zero or a very low minimum look attractive. A production comparison needs a second axis: what happens during a burst?

The autoscaler uses utilization and a target to calculate a target number of replicas. Google documents the form as Ceil(current replicas × current utilization ÷ target utilization). It evaluates every 15-second cycle and chooses the highest target from the previous five-minute window. ^[2] This gives the service a way to react quickly to high observed utilization, but it does not mean a newly requested node is ready in 15 seconds.

The delay after the scaling decision includes provisioning the VM, downloading the container, and loading the model from storage. ^[2] The duration depends on image size, model size, region capacity, accelerator availability, and application initialization. Those are workload variables, so a generic cold-start price or latency should not be invented from the list price.

Use a small load test to measure three quantities: steady-state requests per replica, time to first successful request after scale-up, and the latency percentile during the scale event. Then choose the minimum replicas that cover the traffic you cannot afford to queue. If a morning burst takes several minutes to warm and your SLO is seconds, the apparent autoscaling saving may be economically false because it buys a cheaper bill by violating the service requirement.

Throughput also changes the economics of a larger machine. One larger node may cost more per hour but serve more concurrent requests, while two smaller nodes may cost more or less depending on the published rates and failure-domain needs. Compare configurations on cost per successful request at the required p95 latency, not on hourly rate alone. For broader GPU capacity comparisons, the site’s GPU inference TCO calculator provides a useful adjacent model, while this guide focuses on Vertex AI endpoint node-hours.

Worked cost example

The following is an inspectable estimate, not a Google quote for every endpoint. It uses the $0.0909886 per node-hour rate shown in Google’s published prediction calculation example. ^[1] The example assumes one node, a 31-day month, and no separate accelerator, network, logging, storage, or model-token charges. Your actual rate can differ by region, machine type, accelerator, deployment class, and discount eligibility.

Assume a measured endpoint has this schedule:

The arithmetic is 500 + (2 × 180) + (3 × 64) = 1,052 node-hours, followed by 1,052 × $0.0909886 = $95.72, rounded to cents. The table is a visual demonstration of the method: traffic assumptions become replica-hours, replica-hours become node-hours, and node-hours become a dated estimate.

Now compare the same three-replica capacity held warm all month. That fixed pool uses 3 × 744 = 2,232 node-hours, or about $203.09 at the same illustrative rate. The autoscaled schedule is about $107.37 lower in this simplified serving-resource model. A one-replica fixed pool would be about $67.70, but it is not an equivalent throughput or latency configuration; it cannot be compared as an apples-to-apples replacement for the three-replica burst capacity.

This comparison shows why average replicas matter more than the maximum alone. A maximum of three does not mean you pay for three all month. Conversely, a maximum of 100 does not save money if the autoscaler regularly reaches 100. Record the observed distribution of active replicas, not only the configuration fields.

The example also excludes cold-start impact. If the 64 burst hours occur in short, sharp events and replicas take too long to initialize, you may deliberately raise the minimum before the burst, schedule a warm pool, or redesign the serving container. That can increase endpoint cost while lowering queueing and failed-request cost. Treat the choice as an SLO trade-off, not as a promise that autoscaling always reduces total business cost.

A repeatable budgeting workflow

Start with the deployed model inventory. Record endpoint region, machine type, GPU or TPU type and count, minimum and maximum replicas, model version, and whether the endpoint is shared, dedicated, legacy, or a current deployment class. The deployment guide notes that resources are associated with the deployed model, so two models on one endpoint can still create separate compute decisions. ^[4]

Next, collect a traffic shape instead of one monthly average. Use at least quiet, normal, peak, and burst windows. For each, record requests per second, concurrency, input size, output size where model-token pricing applies, and the measured active-replica range. Averages hide the very peaks that force overprovisioning.

Then convert each window into node-hours. If a window has r average replicas and h hours, its contribution is r × h. Sum all windows. If you have node-running seconds, divide by 3,600 rather than rounding every window. Google’s pricing page says N1 resources are billed in 30-second increments, so second-level telemetry is better than a coarse daily estimate. ^[1]

Apply the correct price table last. Keep on-demand, Spot, reservation, and commitment assumptions in separate columns. Do not mix a discounted rate into a baseline that is meant to describe on-demand exposure. For model pricing that is separate from infrastructure, use the site’s model pricing comparison and keep token meters out of the endpoint node-hour subtotal.

Finally, reconcile the estimate to the bill. Compare predicted node-hours with billing export SKUs and compare predicted replicas with endpoint monitoring. Differences can come from startup and shutdown lag, idle-ready time, traffic bursts, redeployments, accelerator billing, or a rate table that changed. Use the variance to improve the next month’s assumptions rather than silently changing the formula.

FAQ

Does Vertex AI charge for an idle endpoint?

It can. Google’s pricing explanation includes time when a node is in a ready state for online predictions, not just time processing a request. ^[1] If the deployed model keeps its minimum replicas ready, those node-hours belong in the estimate even during a quiet period. Undeploying or deleting a model is a lifecycle action, not the same as setting a low autoscaling maximum.

What is the Vertex AI endpoint cost per hour?

There is no single universal endpoint cost per hour. The applicable price depends on region, prediction type, machine type, accelerator count, deployment class, and discount or commitment terms. Google’s published tables include different rates and describe the unit as a node hour. ^[1] Use the exact row for your deployed configuration and multiply by active nodes; do not multiply a token price by uptime or assume a GPU price includes every management fee.

Can a Vertex AI endpoint scale to zero?

Not every endpoint can. The pricing documentation says legacy MLS1 machine type deployments can scale down to zero for no-traffic periods, while current Gemini Enterprise Agent Platform deployments and other deployment types cannot. ^[1] Confirm eligibility in the documentation and API for your deployment before using zero in a forecast. Even where scale-to-zero is supported, a cold-start test is necessary before accepting the latency trade-off.

How should I budget minimum replicas?

Budget the minimum as a warm baseline: minimum replicas × deployed hours × applicable node-hour rate. Add extra node-hours for the windows where traffic drives the endpoint above the minimum. Use a 30-day or 31-day calendar assumption explicitly, then replace it with billing-export data after launch. Choose the minimum from measured latency, concurrency, and failure tolerance rather than from a cost target alone.

Does a larger machine always reduce endpoint cost?

No. A larger machine may handle more concurrent work or reduce latency, but it can have a higher hourly price and may be underutilized. Google’s compute guidance says machine types differ in vCPU, memory, and price and that more resources can improve latency or concurrent request capacity. ^[3] Benchmark at the target p95 latency and compare cost per successful request. If your deployment also uses token-based model charges, check those separately; the AI token cost calculator is useful for that other meter.

What causes the cold-start delay?

The documented contributors include provisioning and starting the VM, downloading the container, and loading the model from storage. ^[2] Image size, model size, initialization code, region capacity, and accelerator availability affect the observed result. Measure the delay with the same container and model artifact you plan to run. A list-price worksheet cannot infer this latency from node-hours.

Limitations and assumptions

This guide’s arithmetic is an estimate based on node-hours. It does not quote a guaranteed future price, and it does not predict traffic, rankings, impressions, savings, or business outcomes. Pricing and product names can change; the observations above are dated 2026-11-02. Check the live Google Cloud pricing table and endpoint documentation before committing to a budget.

The worked example uses one published illustrative rate and a constructed replica schedule. It excludes taxes, credits, committed-use discounts, Spot interruptions, storage, networking, logging, monitoring, support, model-token usage, batch jobs, and downstream application costs. It also assumes the replica counts are active for the stated hours and does not model provisioning lag separately. Replace every assumption with measured telemetry for a production forecast.