Research-backed calculator guide

Vertex AI Model Garden Managed API vs Deployed Endpoint Cost

Compare Vertex AI Model Garden managed API and deployed endpoint costs with token, node-hour, replica, utilization, and operational assumptions.

What the two Model Garden paths actually mean

Model Garden is a catalog and workflow layer. Google says it brings Google, partner, open-source, foundation, fine-tunable, and task-specific models into one place, with discovery, testing, tuning, and deployment workflows. It also says serving can be handled through the platform, but that does not mean every model has the same commercial path. ^[1]

A managed API path is a provider-operated interface. Partner models are explicitly offered as managed APIs, and Google publishes token-based rates for models such as Claude, Llama, Qwen, Mistral, and others. For example, the partner-model price table observed on November 2, 2026 lists Llama 3.3 70B at $0.72 per million input tokens and $0.72 per million output tokens. ^[3] This is a useful example rate, not a universal Model Garden price. Model, region, context length, batch mode, caching, promotions, and future changes can alter the bill.

A deployed endpoint is a serving configuration you choose. The Model Garden deployment instructions ask you to select a location and a machine type for each node; they also show that you can use a reservation and customize deployment settings. ^[5] The endpoint path is appropriate when you need a deployable open model, a custom container or weights, a fixed serving configuration, specialized accelerators, or control over the serving lifecycle. It also transfers more cost responsibility to your team: selecting hardware, monitoring utilization, keeping the endpoint available, and removing unused deployments.

Do not assume that “managed API versus deployed endpoint” is a choice available for every card. Start with the model card and its supported deployment options. If the model is only exposed as an API, compare token meters and related features. If it is deployable, obtain the machine and accelerator requirements before doing any break-even math. A token rate without a capacity test is not a serving plan.

The meters to put in one comparison

Managed API cost begins with the request mix. Record input tokens and output tokens separately because a model can charge different rates for them. Also record cache hits, long-context thresholds, batch discounts, grounding calls, and any provisioned-capacity commitment. The Google pricing pages state that generative AI prices are in USD and that some model features have distinct meters. ^{[2] [3]}

Endpoint cost begins with capacity. The relevant unit is usually a node-hour: one node running for one hour. Google’s prediction pricing documentation says that an active endpoint with one or more deployed models can incur node-hour charges while the node is processing requests and while it is waiting in a ready state. It also describes fractional time billing and notes that prediction usage is billed in 30-second increments for the applicable machine resources. ^[4]

That ready-state detail is the source of the common “Vertex deployed model idle cost” surprise. A request-metered API can be quiet and produce no token charges. A deployed endpoint can be quiet while still holding a serving node ready for the next request. Google’s platform pricing documentation says standard Vertex deployments cannot scale down to zero nodes, while legacy exceptions are a separate case. ^[4] Treat any scale-down assumption as something to verify for the exact deployment type, rather than as a default.

Count replicas, not just endpoints. Two replicas at $0.20 per node-hour are $0.40 per hour before other costs. A second region doubles the capacity calculation unless traffic is split and the rates differ. A GPU or TPU deployment may be necessary for the model, and its price can dominate a CPU example. You also need to test whether the selected node can serve the required tokens per second. A cheap node that cannot meet peak concurrency is not a cheaper production design.

Worked cost example

The table below is an original, inspectable example. It is deliberately small enough to audit with a calculator and does not claim to be a recommendation for a large language model.

The managed calculation is 20 × 0.72 + 5 × 0.72, where the token quantities are in millions. The endpoint calculation is 1 × 720 × 0.0770564. The month is a planning convention of 30 days × 24 hours, not a promise about an invoice period. If the endpoint runs for 744 hours in a 31-day month, the same one-node example becomes $57.33. If you keep two replicas for high availability, the 720-hour endpoint cost becomes $110.96.

The break-even result needs careful interpretation. At this illustrative managed blend, the API costs $18 / 25M = $0.72 per million tokens. Dividing $55.48 by $0.72 gives 77.06 million tokens. That is only a raw meter comparison. It assumes one node can process 77 million tokens in the month, including bursts, without adding replicas. If the model needs a GPU, if the node cannot meet p95 latency, or if traffic is concentrated into a few hours, the break-even moves. If the managed API rate changes, the break-even moves too.

A more useful endpoint calculation uses utilization and throughput. Suppose a benchmark shows one deployed node can generate 100 tokens per second at the required quality and latency. The theoretical monthly output ceiling is 100 × 60 × 60 × 720 = 259.2M tokens, before headroom and input processing. A 77.1M-token workload would use about 29.7% of that theoretical output capacity. If the benchmark is 20 tokens per second, the same workload would exceed the ceiling. In production, reserve headroom for peak concurrency, retries, health checks, deploys, and a safety margin; never divide by average throughput alone.

How to build a defensible Vertex AI MaaS vs endpoint pricing sheet

First, identify the billing shape from the model card. Mark the candidate as managed API, deployable endpoint, or both. Google’s documentation provides a deployment workflow and lets you inspect deployment specifications, including machine type, accelerator type, and container information, before deploying. ^[5]

Second, freeze the workload window. Use one month, one region, one model version, and one traffic profile. Record average and p95 input tokens, output tokens, requests per minute, peak concurrency, and expected idle hours. A monthly average hides the difference between steady traffic and a four-hour daily batch.

Third, calculate managed cost with the actual input/output mix. For a managed partner model, use the model’s current table row. For a Google model, use its current row on the generative AI pricing page. Add distinct charges for caching, grounding, batch, provisioned throughput, or other features only when your workload uses them. Keep promotional rates separate from standard rates and write the end date beside any temporary price.

Fourth, calculate endpoint cost from the deployment specification. Use replicas × hours × node-hour rate, then add any applicable accelerator, management, storage, data-transfer, logging, registry, or networking charges. The official Vertex pricing page explains that Spot VMs and Compute Engine reservations can have their own billing treatment and that Vertex inference management fees can sit alongside infrastructure usage. ^[4] Do not hide those items inside a single “GPU price” unless your quote explicitly bundles them.

Fifth, add ownership cost as a scenario rather than pretending it is zero. Endpoint ownership may include model packaging, container updates, on-call coverage, autoscaling tests, capacity reservations, security review, observability, incident response, and undeployment controls. These are estimates that you must supply from your team’s labor rates. They are not included in the Google node-hour example above.

Finally, run sensitivity cases. Test one and two replicas, 30-day and 31-day months, low and peak traffic, a regional fallback, and at least two throughput results from a real benchmark. If the endpoint wins only at 95% utilization, call that fragile. If the API wins while the endpoint is idle half the day, call out the idle penalty. The right answer is a workload-specific range, not a universal “self-hosting is cheaper” slogan.

When the managed API is usually the better starting point

Choose the managed path when demand is intermittent, the model is available at a reasonable token rate, the workload is still being validated, or the team does not yet have a trustworthy throughput benchmark. It also fits teams that value a single API surface, Google Cloud billing integration, and less responsibility for endpoint lifecycle management. “No endpoint” does not mean “no bill”: tokens, context, caching, grounding, and other enabled features still matter.

The managed path is especially attractive when your monthly token spend is far below the cost of keeping even one suitable replica warm. It avoids paying a full month of capacity for a small test and makes scale driven by requests rather than by a manually maintained node count. You should still budget quotas, retries, logging, network egress, and application overhead.

Use the Model pricing comparison page to normalize input and output rates before comparing providers. If the workload includes retrieval, add the RAG, embedding and vector database cost calculator rather than treating embedding and vector serving as free side effects.

When a deployed endpoint can be justified

A deployed endpoint can make sense when the model is not exposed through the managed API you need, when a custom or open model is central to the product, or when a measured capacity plan keeps replicas busy enough to amortize their hourly cost. It may also be justified by latency, data placement, customization, or predictable serving requirements that a request-priced API cannot meet.

The cost case becomes stronger when the model fits a smaller machine, the workload is steady, and throughput is high enough to keep the serving layer productive without breaching the SLO. It becomes weaker when the endpoint is left running for experiments, when autoscaling cannot remove the minimum capacity, or when an oversized accelerator is selected “just in case.” Use the Self-hosted GPU inference cost calculator to model accelerator capacity, utilization, and cost per million tokens rather than copying a generic hourly number.

Deployment also creates an operational control problem. Put an owner and expiration date on every test endpoint. Review replicas and regions in billing reports. Alert on node-hours, not only token usage. When a model is no longer serving traffic, undeploy it according to your change-control process; a stopped application does not necessarily mean the endpoint has released its serving capacity.

Assumptions, exclusions, and limitations

All dollar values in the worked table are USD observations or calculations dated 2026-11-02. The $0.72 per million token rate is the cited Llama 3.3 70B partner-model example, not a guarantee for another model, region, context length, batch mode, or date. The $0.0770564 node-hour figure is the cited E2 standard-2 price shown for the selected official pricing region; a deployable open model may require a different machine or accelerator.

The example excludes taxes, committed-use discounts, reservations, Spot pricing, storage, model artifact transfer, network egress, logging, monitoring, security tooling, quotas, failed application retries, support plans, and engineering labor. It also excludes the cost of tuning and evaluation. It assumes all 25 million tokens are billed normally and that the managed API has no cache or batch adjustment.

No search-volume, keyword-difficulty, traffic, ranking, indexing, revenue, or advertising outcome is inferred from the SERP benchmark. Pricing pages change. Re-check the exact model card, region, SKU, and contract terms immediately before committing spend. Treat the table as a method for making your own assumptions explicit, not as a vendor quote.

FAQ

Is a Vertex AI Model Garden managed API billed like a deployed endpoint?

No. A managed API generally meters requests through model-specific units such as input and output tokens. A deployed endpoint meters serving capacity, commonly node-hours, including time a node is ready for online prediction. Some deployments can also involve separate storage, networking, accelerator, or platform charges. Confirm the exact model and SKU before comparing.

Do I pay for an idle Vertex AI deployed endpoint?

You can incur charges while serving nodes remain ready even when request volume is low. Google’s pricing documentation describes node time in a ready state as part of prediction billing and says standard Vertex deployments cannot scale to zero. The exact behavior depends on the deployment type and configuration, so verify it before relying on an idle-hours assumption. ^[4]

What is the simplest managed API versus endpoint break-even formula?

Use monthly endpoint capacity cost ÷ managed cost per million tokens. For the worked example, $55.48 ÷ $0.72 = 77.1M tokens. Then validate that the endpoint’s tested throughput, peak concurrency, replica count, and latency target can actually serve that volume. The formula is a screening tool, not a capacity guarantee.

Is a deployed endpoint always cheaper at high volume?

No. High volume improves endpoint utilization, but the result still depends on the required machine, accelerator, replica count, region, throughput, quality target, and operational costs. A managed API can remain cheaper if it has a low token rate or if your endpoint needs expensive hardware and redundancy. Compare effective cost per successful request, not only raw node-hours.

Can every Model Garden model be used through both paths?

No. Availability differs by model. Google’s Model Garden pages distinguish managed partner APIs from open or fine-tunable models that can be deployed, and the deployment instructions require you to inspect supported options and machine specifications. Start with the model card rather than assuming that an API model can be self-hosted or that an open model has a managed token endpoint. ^{[1] [3] [5]}