Research-backed calculator guide

Together AI Serverless vs Dedicated Endpoints: Model the Idle-Replica Crossover

Compare Together AI serverless token charges with dedicated endpoint GPU-minute costs, then calculate the traffic level where a ready replica can break even.

serverless monthly cost
  = input millions × input price + output millions × output price

dedicated monthly cost
  = ready replica hours × GPU-hour rate × GPUs per replica

For the worked assumptions below, an H100 DMI replica running 12 hours per day costs $1,976.40 over a 30-day month. Using a serverless model observed on Together’s pricing page at $0.32 per million input tokens and $1.28 per million output tokens, with an 80% input / 20% output mix, the blended rate is $0.512 per million tokens. The crossover is about 3.86 billion total tokens per month. That is an estimate, not a Together quote: it excludes reserved-capacity terms, taxes, retries, non-inference services, and any difference between your model’s throughput and this example.

What Together AI bills in each deployment mode

Serverless: a usage meter with no idle-replica floor

Together’s serverless overview describes serverless inference as an API with per-token pricing for chat, language, embedding, and rerank workloads. Image, video, and speech products use their own units, such as megapixels or seconds. The key cost property for this article is that serverless has no minimum cost and no provisioning cost: if you send no token-bearing requests, there is no dedicated replica sitting ready solely for your account. ^[2]

The serverless formula for a text model is therefore:

C_serverless = (I / 1,000,000 × P_in) + (O / 1,000,000 × P_out)

I is input tokens for the month, O is output tokens, and the two prices are dollars per million tokens. Keep input and output separate. A workload with short prompts and long completions can have a very different cost from a retrieval-heavy workload with long prompts and short completions, even when total tokens are identical.

The rate is model-specific and can change. As a dated observation for this example, Together’s pricing page showed Qwen3.7-Plus at $0.32 per million input tokens and $1.28 per million output tokens on 2026-11-02. Verify the selected model and current rate before using the result for a purchase decision. ^[1]

Dedicated Model Inference: ready-replica time

DMI changes the meter. Together’s dedicated pricing documentation says a deployment bills by the minute, each running replica bills independently, and billing applies only while a replica is ready and able to serve traffic. Provisioning or cold-start time is not billed, and a replica stops billing as soon as it scales down. ^[3]

This makes the practical DMI formula:

C_DMI = Σ (ready replicas × minutes ready × dollars per GPU-minute × GPUs per replica)

A minReplicas value creates the cost floor for a deployment. A maxReplicas value limits the ceiling. If both bounds are zero, or the deployment is stopped, Together says it bills nothing while stopped. An inactivity timeout can also stop a deployment after an idle period, but the resulting cold start is a service-level trade-off rather than a free capacity guarantee. ^[3]

The GPU type and count matter. Together’s documentation listed a single H100 80GB at $5.49 per hour when checked on 2026-11-02; a multi-GPU configuration costs proportionally more, and hardware without a listed public price requires a sales quote. ^[3] The product overview also describes DMI as reserved hardware with autoscaling and the same inference APIs as serverless, so moving an application does not necessarily require rewriting its request integration. ^[4]

A repeatable Together AI crossover method

Start with a billing period and an actual traffic profile. Monthly averages alone hide the idle periods that drive DMI economics. Record at least:

1. Input and output tokens by month, ideally split by model.
2. The input/output mix, because the two serverless rates differ.
3. The number of GPUs per replica and the observed H100, B200, or quoted rate.
4. The number of ready replicas by hour or by traffic band.
5. The minimum and maximum replica bounds, plus any scheduled stop periods.
6. The throughput and latency headroom required for peak traffic.

Then calculate two independent totals. For serverless, multiply each token class by its per-million rate. For DMI, multiply the ready time of every replica by the applicable per-GPU-hour rate. Do not multiply maxReplicas by all hours unless monitoring shows that all replicas are actually ready for all of those hours. Conversely, do not use an optimistic average if minReplicas forces a replica to remain ready during quiet periods.

The simple crossover equation is:

break-even tokens
  = dedicated monthly cost ÷ serverless blended dollars per million × 1,000,000

For a mixed workload, calculate the blended rate as:

blended rate = input share × input price + output share × output price

This is only valid if the shares describe the same total token pool and the selected model has one stable rate for the comparison period. If your traffic uses several models, calculate each model separately or use a weighted rate tied to measured usage.

Worked cost example: one H100 replica with a daily active window

Assume a team is comparing serverless with one DMI replica for a text workload. The team needs the replica ready for 12 hours per day, but it can stop the deployment for the remaining 12 hours. It uses one H100 GPU per replica. The traffic mix is 80% input and 20% output. The example uses the dated Qwen3.7-Plus rates observed on Together’s pricing page, not a guaranteed future price.

At 1.5 billion monthly tokens, serverless is lower by $1,208.40 in this simplified model. At 4.0 billion tokens, the 12-hour DMI schedule is lower by $71.60. The crossover is approximately 3.86 billion tokens per month, provided the replica can process the traffic within the 360 ready hours and the workload does not require a second replica during bursts.

The most important sensitivity is not just token volume. If the H100 is ready continuously for 30 days, the DMI cost becomes $3,952.80, and the same blended workload crosses over at about 7.72 billion tokens. If peak concurrency forces two ready replicas for part of the day, add the second replica’s ready minutes. If the deployment can reliably scale to zero between windows, remove those stopped hours; if minReplicas: 1 keeps one replica ready, those floor hours remain in the calculation. Together’s documented rule is ready-replica time, not a vague “dedicated endpoint fee.” ^[3]

How to model idle replicas and burst headroom

An idle-replica crossover is a capacity question as much as a pricing question. A replica may be handling few tokens while still being valuable for latency, predictable throughput, or a peak event. Treat that value as an explicit operating requirement rather than quietly calling idle time waste.

Use an hourly or fifteen-minute usage export if available. For each interval, estimate the replicas required to meet the latency and throughput target. Multiply that count by interval length and the GPU rate. This produces a measured ready-replica cost rather than a monthly average that can conceal a large night-time floor.

For burst headroom, model at least three bands: normal traffic, a high-percentile peak, and a failure or retry case. maxReplicas limits the financial exposure but also limits the capacity available at the peak. A lower maximum can make the budget look attractive while increasing queuing or errors. The Together overview says DMI autoscaling can scale deployments with replicas and the pricing documentation explains that minReplicas and maxReplicas are the principal floor and ceiling controls. ^{[3] [4]}

Serverless also has operational limits. Together notes that response times can vary with model and demand and that high request volumes or large bursts can sometimes encounter rate limiting. If a serverless estimate is cheaper but cannot meet the required tail latency or burst behavior, the cheapest invoice is not the cheapest feasible design. ^[2]

Reserved capacity, multi-GPU replicas, and exclusions

Together documents on-demand DMI as minute-based with no commitment, while reserved capacity is a lower effective rate for a committed term with guaranteed hardware availability. The public pricing page does not provide one universal reserved quote for every workload. Use the actual quote, commitment term, cancellation terms, and region-specific conditions in a separate scenario; do not substitute the on-demand H100 rate for a reserved price and call it a reservation forecast. ^[3]

A multi-GPU replica multiplies the hardware portion of the formula. If a single replica uses four GPUs, a four-GPU hourly rate is approximately four times the single-GPU rate where Together publishes proportional pricing. The model may require that configuration to fit in memory or achieve the needed throughput, so compare complete configurations rather than comparing GPU names alone. ^[3]

This guide’s arithmetic excludes tokens from retries unless they are included in the measured token total, and it excludes storage, logging, networking, taxes, support agreements, fine-tuning, provisioned throughput, and application infrastructure. It also excludes the value of engineering time, latency improvements, and downtime avoided by reserved capacity. Those belong in a broader total-cost model. For a fuller model-versus-model comparison, use the Model pricing comparison calculator, then validate measured tokens with the AI token cost calculator. Reconcile the estimate against provider usage and invoices with the AI API invoice reconciliation calculator.

FAQ

Is Together AI serverless cheaper than a dedicated endpoint?

Not in every workload. Serverless is usually cost-favorable when traffic is low or bursty because it does not charge for an idle dedicated replica. DMI can become cheaper when a replica is ready for many hours and processes enough tokens to spread the fixed GPU-minute cost. Calculate both using your actual input/output mix, ready-replica schedule, and required peak capacity.

How is Together AI dedicated endpoint cost calculated?

Together bills DMI by the minute for each ready replica, based on the hardware configuration. A replica that is provisioning, cold-starting, degraded, or scaled down is not charged under the documented rules. Multiply ready minutes by the per-GPU-minute rate and by the number of GPUs in the replica, then sum across replicas and deployments. ^[3]

What is the Together AI GPU-minute billing formula?

Use ready replicas × ready minutes × GPUs per replica × dollars per GPU-minute. Convert an hourly price by dividing by 60. For the example H100 rate, $5.49 per hour equals $0.0915 per GPU-minute. This is a billing calculation; it does not promise that one H100 can serve a particular token volume or latency target.

Does scaling to zero make a dedicated endpoint free?

When a deployment is scaled to zero or stopped, Together’s documentation says it bills nothing while stopped. However, scaling to zero can introduce a cold start when traffic returns. Model the stopped hours as zero DMI cost only if the resulting startup delay is acceptable for the workload. ^[3]

What does a reserved-capacity break-even calculation need?

It needs the actual reserved rate, committed duration, minimum capacity, hardware availability terms, and any usage or termination conditions in the quote. Replace the on-demand DMI cost in the crossover formula with the committed cost allocated to the same period. Keep a separate on-demand scenario so the commitment premium or discount is visible.

Which number should I use for serverless tokens?

Use measured input and output tokens for the exact model and period. Do not use request count as a token proxy unless you have a defensible tokens-per-request distribution. If prompt size, completion length, or model mix changes, recalculate the blended rate instead of carrying forward the old crossover.