Research-backed calculator guide

SageMaker Serverless vs Real-Time Inference Cost: Add Provisioned Concurrency Only When It Pays

Compare SageMaker serverless, real-time endpoint, and provisioned-concurrency costs with one workload model, explicit assumptions, and a break-even method.

What the three choices actually meter

On-demand Serverless Inference

On-demand serverless has two important cost dimensions. First, inference duration is tied to the memory configuration and the time spent processing requests. Second, AWS charges for data processed into and out of the endpoint. The pricing page describes the duration meter as billed by the millisecond and says the compute charge depends on the selected memory configuration. ^[1]

That makes idle time cheap in a narrow billing sense: if the endpoint receives no request, there is no request duration to meter. It does not make the endpoint free. Data processing, monitoring, logging, downstream services, and any other architecture costs remain outside this simple request-duration calculation. Serverless also has performance constraints. A cold start can occur after inactivity or when concurrent demand exceeds warm capacity, so the cheapest meter can be a poor fit for a strict first-request latency objective.

Real-time inference

A real-time endpoint uses the SageMaker instance type selected for hosting. AWS's pricing description says customers are charged for usage of that instance type. ^[1] In a normal always-ready design, the endpoint's instance-hours continue while the endpoint is deployed, including periods when utilization is low. That is the core SageMaker endpoint idle cost: capacity is being held for readiness rather than purchased only for the milliseconds that a request executes.

Auto scaling can change the shape of that bill by adjusting instance count. It does not remove the need to model a minimum capacity, scale-out lag, warm-up behavior, and peak capacity. Compare like with like: a single serverless request-duration estimate should not be compared with an always-on endpoint sized for peak traffic unless the model explicitly includes both the quiet hours and the busy hours.

Serverless with Provisioned Concurrency

Provisioned Concurrency keeps a configured number of serverless execution environments ready. AWS describes it as a way to mitigate cold starts and provide more predictable performance; the service can also scale that setting with Application Auto Scaling or a schedule. ^[2] The cost has at least three logical lines:

1. Warm-capacity charge: configured concurrency × provisioned seconds × the applicable memory-based rate.
2. Request-duration charge: the time spent serving requests, using the applicable rate for requests handled by the provisioned capacity.
3. Data processing: bytes processed in and out of the endpoint.

The first line is the one most often omitted from a superficial SageMaker provisioned concurrency pricing comparison. If you configure 100 warm slots for eight hours, you pay for 800 slot-hours whether the requests fill every slot or arrive in short bursts. Autoscaling can help. AWS documents target tracking through SageMakerVariantProvisionedConcurrencyUtilization and scheduled actions for predictable traffic windows. ^[3]

A practical break-even model

Use one month of observed or forecast traffic. Define:

• R = requests in the month.
• t = average billed inference duration per request in seconds. Use a percentile or a conservative measured duration when latency and cost are sensitive.
• M = configured serverless memory.
• D = GB processed in plus GB processed out.
• h = hours in the month that the real-time instance is deployed.
• n = average real-time instance count, including the quiet period.
• p_rt = hourly price of the selected real-time instance in the selected Region.
• p_od(M) = on-demand serverless duration rate for memory M.
• p_pc = provisioned-concurrency warm-capacity rate for the selected memory and Region.
• c = provisioned concurrency count.
• w = hours for which that concurrency is provisioned.
• p_pc_req = request-duration rate for requests served by Provisioned Concurrency.
• p_data = applicable data-processing price per GB.

The simplified estimates are:

On-demand serverless
= (R × t × p_od(M)) + (D × p_data)

Real-time endpoint
= (n × h × p_rt) + (D × p_data)

Serverless with Provisioned Concurrency
= (c × w × 3,600 × p_pc)
  + (requests served while warm × t × p_pc_req)
  + (requests served on demand × t × p_od(M))
  + (D × p_data)

These formulas intentionally exclude items that are not common to the three paths, such as training, model artifact storage, data transfer outside the endpoint, CloudWatch charges, downstream databases, taxes, and enterprise discounts. Add those separately when they differ by design.

Worked cost example

The following table is an original calculation using a 30-day, 720-hour month. The serverless rates are the dated rates shown in AWS's published pricing examples for a 2 GB configuration: $0.00004 per second for on-demand inference, $0.000023 per second for inference served by Provisioned Concurrency, $0.000010 per provisioned-concurrency second, and $0.016 per GB of data processed. AWS's example also illustrates that Provisioned Concurrency and inference duration are separate components. ^[1]

The real-time comparison uses the $0.204/hour ml.c5.xlarge rate shown in AWS's published hosting example. Treat it as an illustrative, dated input rather than a universal current quote. ^[1] The model receives 3 million requests, each taking 100 ms, and processes 3 GB total in and out. No free tier, Savings Plan, tax, or discount is applied.

The example gives three different answers to three different operating goals. On-demand serverless is the lowest modeled bill because the workload has only 300,000 seconds of request time. Real-time costs more because it buys one instance's availability for the full month. The hybrid path costs more than pure on-demand serverless because it deliberately buys four warm slots for a defined 40-hour window, but it remains far below the illustrative always-on instance bill.

The same arithmetic also shows why “add Provisioned Concurrency” is not a blanket recommendation. If the team changed the warm window to 720 hours and the warm count to 100, the warm-capacity line alone would be 100 × 720 × 3,600 × $0.000010 = $259.20, before request duration and data. That may be justified for a latency-sensitive service, but it is not a saving simply because it is serverless. Reduce the count, schedule it around demand, or keep on-demand behavior when the latency requirement permits cold starts.

For a second sanity check, calculate the real-time utilization threshold from the same inputs. At $0.204/hour, one instance costs $146.88 for 720 hours. Ignoring data charges, on-demand serverless reaches that amount at 146.88 ÷ 0.00004 = 3,672,000 billed seconds. At 100 ms per request, that is about 36.72 million requests. This is not a deployment recommendation or a universal crossover. It changes with memory, request duration, instance family, concurrency, Region, scaling policy, and the number of real-time instances. It simply makes the assumption visible.

How to decide without fooling yourself

First, measure traffic by hour rather than using a monthly average. A workload that averages 10 requests per minute may still have a short burst that requires warm capacity. Record request count, duration, concurrency, cold-start occurrences, payload bytes, and the endpoint's memory setting. AWS's Provisioned Concurrency guidance specifically points to utilization metrics and scheduled or target-based scaling for tuning the setting. ^{[2] [3]}

Second, separate a performance constraint from a cost preference. If a cold start violates the first-token or first-response SLO, calculate the smallest warm capacity that covers the affected traffic window. Provision only that window. If the SLO is measured at a high percentile, size from observed concurrency at that percentile rather than from the average.

Third, compare the complete real-time shape. Include the minimum instance count, expected scale-out time, redundancy count, and hours at each instance level. A two-instance production endpoint is not a fair comparison with a one-slot serverless configuration. Conversely, a serverless endpoint that needs high memory or high concurrency may not be an appropriate substitute for the selected real-time instance.

Fourth, test operational limits before optimizing the bill. The official Serverless Inference documentation and configuration guides should be the source of truth for supported memory, concurrency, payload, model, Region, and container behavior at deployment time. ^[4] Real-time endpoints have their own instance, scaling, and deployment constraints. ^[5] A lower estimate is irrelevant if the model does not fit the selected deployment mode or the endpoint cannot meet the response contract.

Finally, recheck prices at the decision date. The AWS pricing page is the authoritative place to verify the Region-specific rates and free-tier treatment. ^[1] A calculator can make arithmetic repeatable, but it cannot remove the need to choose the correct instance family, memory tier, traffic window, data meter, and account assumptions.

If the workload also includes retrieval, embeddings, or a vector database, keep those costs separate from endpoint hosting and use the RAG, embedding and vector database cost calculator. For a broader model-provider comparison, see the Model pricing comparison. If inference is one component of a larger build, the AI agent development cost estimator can keep engineering and operating costs in the same planning conversation.

FAQ

Is SageMaker Serverless Inference always cheaper than a real-time endpoint?

No. It can be cheaper for intermittent traffic because request duration replaces an always-ready instance in the main compute meter. A real-time endpoint can be cheaper at sustained utilization, at high concurrency, or when the serverless memory and duration charges exceed the price of the instance capacity required. Compare the same model, request duration, memory requirement, Region, data volume, and redundancy assumptions. Do not use a fixed request-count rule without those inputs.

Does a real-time SageMaker endpoint charge when it is idle?

A deployed real-time endpoint is billed for the instance type and duration of use, not only for seconds containing a successful request. If autoscaling retains a minimum instance, that minimum creates an idle-capacity cost during quiet periods. The exact bill depends on instance count, scaling actions, Region, and any associated services. AWS's pricing page describes real-time inference as charging for the selected instance type. ^[1]

What does SageMaker Provisioned Concurrency add to the bill?

It adds a warm-capacity charge based on configured memory, the duration provisioned, and the concurrency count. Requests still incur inference-duration and data-processing charges. ^[1] A useful monthly model therefore has a warm-capacity line and request-duration lines; it should not treat Provisioned Concurrency as a free switch that only changes latency.

When should I schedule Provisioned Concurrency instead of leaving it on?

Schedule it when latency-sensitive demand is predictable, such as business hours or a known event window. AWS documents scheduled scaling as an option for routine traffic and target tracking as an option for utilization-driven changes. ^[3] Start with the smallest concurrency that protects the measured latency objective, then compare its warm-capacity charge with the cost of on-demand serverless plus the operational impact of cold starts.

Can I use the AWS pricing-page example as my production quote?

No. Use it as a transparent arithmetic reference. AWS pricing examples show how the meters work, but your quote depends on Region, memory, instance family, request duration, data processing, free-tier eligibility, discounts, scaling, and other services. The prices and observations in this article were checked on 2026-11-02; verify them again before deployment.

What costs are excluded from this comparison?

The worked example excludes model training, model artifact storage, ECR or container costs, CloudWatch logs and metrics, data transfer outside the stated endpoint data meter, API Gateway or application infrastructure, multi-AZ redundancy beyond the modeled instance count, taxes, and negotiated discounts. It also excludes the value of engineering time and the business cost of missed latency targets. Add those lines when they differ between the designs.