Research-backed calculator guide

SageMaker Asynchronous Inference Cost: Queue Large Payloads Without Paying for a 24/7 SLA

Learn how to estimate SageMaker Asynchronous Inference cost from instance-hours, cooldown time, storage, request and response bytes, and same-Region S3 payload assumptions.

What SageMaker asynchronous inference changes

A real-time endpoint is designed to answer while the caller waits. Its capacity is normally present even between requests. Asynchronous inference changes the interaction: the caller puts the payload in S3, sends a pointer with InvokeEndpointAsync, and receives a request identifier and output location. SageMaker queues the work and writes the prediction to S3 when processing finishes ^[2]. An optional Amazon SNS notification can signal success or failure.

That architecture is useful when the input is a document, image, video, audio file, or another large object. The caller can poll or consume a notification instead of holding an HTTP request open. It is less suitable when the product promise requires an immediate prediction or when queue delay is itself a failure.

The important billing boundary is the difference between the S3 object and the invocation envelope. In AWS’s pricing example, the 100 MB S3 input payload is not treated like ordinary endpoint data processing when it is read from S3 and the output is written to S3 in the same Region. The small request and response bodies still attract data-processing charges ^[1]. S3 storage and S3 request charges are separate services and should be added from their own usage records.

The SageMaker async cost formula

Start with the compute line. If i is the number of instances, h is the billed instance-hours, and r is the Region-specific hourly rate, then:

compute = i × h × r

For a scale-to-zero endpoint, h is not automatically “hours while predictions are actively running.” Include the time the endpoint remains provisioned while a queue drains, the cooldown period before scaling in, and any time spent at a higher instance count during a burst. A useful reconstruction is:

h = active processing hours + scale-up and queue-drain hours + cooldown hours + other provisioned hours

If your autoscaling policy runs two instances for one hour, that is two instance-hours even if each instance is only partly utilized. Conversely, a request waiting in S3 does not by itself mean that an instance is billed while the endpoint is at zero.

Add attached endpoint storage:

endpoint storage = provisioned GB × GB-month rate

AWS’s published example uses 4 GB of general-purpose SSD at $0.14/GB-month, or $0.56 for that line ^[1]. Treat this as the rate and unit shown in that example, not a universal current quote. Storage retention for input and output objects in S3 is not the same line.

Finally, estimate billable request and response data:

data processing = (request GB + response GB) × data-processing rate

Use the billing unit shown in the regional price sheet and keep your conversion consistent. Do not silently mix decimal megabytes with binary gibibytes. Log both the raw bytes and the converted billing quantity so a finance reviewer can reproduce the number.

Worked cost example

Consider an endpoint in a Region where the illustrative rates below match the AWS pricing example. The date checked for this article is November 2, 2026. The rates are an example baseline, not a vendor quote for your account.

Assume one ml.c5.xlarge endpoint instance at $0.20/hour, a 30-day month, and a scale-to-zero policy. The endpoint is provisioned for two hours of active work per day and remains up for a 30-minute cooldown. This produces 2.5 × 30 = 75 instance-hours. Attach 4 GB of general-purpose SSD at $0.14/GB-month. Each of 1,024 daily requests has a 10 KB invocation body and a 10 KB response body. Use $0.02/GB for each direction, matching the illustrative pricing example. The large 100 MB input object and its 10 MB output are stored in S3 in the same Region; S3 storage and S3 requests are excluded from this endpoint-only calculation.

The compute line is 75 × $0.20 = $15.00. Endpoint storage is 4 × $0.14 = $0.56. Over 30 days, each direction carries 10 KB × 1,024 × 30 = 307,200 KB, or approximately 300 MB using 1,024 KB per MB. That is approximately 0.293 GB per direction. Request plus response processing is therefore 0.586 GB × $0.02 = $0.0117.

The modeled endpoint subtotal is $15.00 + $0.56 + $0.0117 = $15.5717, or about $15.57 for the month. The number is lower than AWS’s $16.38 published example because this scenario uses 30 rather than 31 days and a slightly different presentation of the same 2.5-hour daily pattern. The calculation is intended to make the inputs inspectable, not to replace the live pricing page.

For comparison, if the same one-instance endpoint stayed provisioned for the full 30 × 24 = 720 hours, compute would be $144.00. Keeping storage and request/response processing constant gives an endpoint subtotal of approximately $144.57. Under these assumptions, scale-to-zero reduces the instance-hour portion by $129.00. It does not guarantee a 9.3-times lower complete AWS bill because S3 retention, notifications, logs, network paths, retries, and high-availability choices are outside this subtotal.

This table is the visual demonstration for the article: every output can be recomputed from the displayed inputs. Replace the hourly and storage rates with the values for your instance type and Region before using it for a budget decision.

How to compare asynchronous and real-time cost

The fair comparison is not “async has no cost while queued.” Compare the same model, instance family, Region, availability target, and traffic trace. For real-time inference, calculate the minimum provisioned floor and the replicas needed for the peak. For asynchronous inference, calculate the instance-hours produced by the queue and autoscaling policy.

AWS’s inference guidance positions real-time inference for low latency and predictable traffic, serverless inference for synchronous but spiky workloads, asynchronous inference for latency-insensitive and cost-sensitive work up to 1 GB, and batch inference for offline jobs without a persistent endpoint ^[3]. That is a workload decision before it is a price decision.

A real-time endpoint may be the better economic choice when requests are steady enough to keep capacity busy, when a scale-up delay would cause retries, or when the service needs multi-AZ availability at all times. Async can be better when traffic arrives in bursts, payloads are large, and a queue can absorb a measured delay. If the workload is fully offline and can run as a finite job, compare batch transform too; paying for a persistent endpoint may be unnecessary.

Measure queue age and completion latency, not only utilization. A cheaper endpoint that leaves jobs waiting beyond the business deadline can create reprocessing, duplicate S3 objects, support work, or missed revenue. Keep those consequences as explicit internal assumptions rather than pretending they are AWS price meters.

Autoscaling details that affect the estimate

AWS documents target tracking for asynchronous endpoints with metrics such as ApproximateBacklogSizePerInstance ^[4]. The target is a policy input, not a promise that every request completes within a specific time. A low target can scale out sooner and increase instance-hours. A high target can reduce capacity but allow queue age to grow.

Scaling from zero needs special attention. AWS supports MinCapacity: 0, and requests received while there are zero instances are queued. Without an additional scale-from-zero policy, the endpoint may wait until backlog crosses the target-tracking threshold. AWS documents an optional policy based on HasBacklogWithoutCapacity to start capacity when a queue exists but the endpoint has no instances ^[4]. Include that behavior in your latency test and in your estimate of retry or timeout risk.

Use measured traces to choose min_capacity, max_capacity, target backlog, scale-out step, and cooldown. Record the number of instance-hours at each capacity tier. A monthly average hides the exact behavior that drives an async bill: short bursts may cause repeated scale-up and cooldown periods, while a sustained queue may keep multiple instances running for hours.

What the endpoint subtotal excludes

An endpoint estimate is not a complete AWS bill. Add S3 Standard or other storage for retained inputs and outputs, S3 request charges, SNS notifications, CloudWatch logs and metrics, KMS requests or keys where applicable, data transfer out of the Region, NAT or private-network costs, and any pipeline or orchestration service. Include multiple production variants and extra instances for availability when they are part of the design.

Also account for failed requests and retries. A retry can repeat inference work and create another output object. If the client cannot safely identify an idempotency key, the business may pay for duplicate processing even though the original request eventually succeeds. Keep a ledger keyed by request ID, input object version, output object, attempt number, and endpoint variant.

Do not subtract free usage unless the account, date, Region, and service terms are confirmed. Free tiers and promotional allowances can change, and they may be shared across a billing account. Likewise, do not apply a Savings Plan discount to this worksheet unless you have a qualifying commitment and know that the particular usage is covered. AWS’s general optimization guidance discusses Savings Plans, rightsizing, and autoscaling, but a discount is not a universal async price ^[3].

For infrastructure context, run the same active-hours and utilization assumptions through the Self-hosted GPU inference cost calculator. If the endpoint is one stage in a retrieval system, separate it from the RAG, embedding and vector database cost calculator. For model or provider alternatives, use the Model pricing comparison rather than mixing model-token prices into the SageMaker hosting line.

FAQ

Is SageMaker Asynchronous Inference cheaper than a real-time endpoint?

It can be, when traffic is bursty and the endpoint can scale down between jobs. The savings come mainly from fewer provisioned instance-hours, not from a free prediction meter. If traffic is steady, if the endpoint must remain available, or if scale-up delay creates expensive retries, real-time can be competitive. Compare instance-hours, queue latency, availability replicas, and downstream charges under the same traffic trace.

How much does a large SageMaker async payload cost?

The payload itself is commonly placed in S3 and referenced by the asynchronous invocation. AWS’s pricing example says that same-Region S3 input and output movement is not charged as ordinary endpoint data processing, while the invocation request and response bodies are processed under the endpoint data-processing meter ^[1]. You still need to budget S3 storage, S3 requests, retention, and any cross-Region or internet transfer. A 1 GB maximum capability is not a 1 GB price; the bill depends on the endpoint, Region, storage, and data paths.

Can a SageMaker asynchronous endpoint scale to zero?

Yes. AWS documents that asynchronous endpoints can use MinCapacity: 0; requests that arrive with zero instances are queued until the endpoint scales up ^[4]. Test the scale-from-zero policy separately. If the endpoint waits for backlog to exceed a target, the first requests in a quiet period may wait longer than expected. A scale-from-zero alarm can reduce that wait but may increase provisioned hours.

What is the simplest SageMaker async endpoint cost formula?

Use instance-hours × hourly instance rate + endpoint storage + billable request/response GB. Then add S3 storage and requests, notifications, logs, networking, retries, and redundancy. Keep the active endpoint subtotal separate from the full pipeline total so that a change in S3 retention does not get confused with a change in model-serving cost.

Does a queued request consume an instance-hour while the endpoint is at zero?

Not for an instance that does not exist, but the request can trigger scale-up or remain queued until a scaling policy provisions capacity. Once instances are provisioned, their billed hours include processing, queue drain, and cooldown according to the service’s billing behavior and your policy. Inspect CloudWatch metrics and the AWS invoice rather than assigning all queue time to compute by assumption.

Should I use asynchronous inference or batch transform?

Use asynchronous inference when you need an endpoint-like interface, requests arrive independently, and a near-real-time queue is useful. Use batch transform when the work is an offline dataset job and you do not need a persistent endpoint. AWS presents batch inference as an option for large offline sets and asynchronous inference as an option for latency-insensitive queued requests ^[3]. Price the actual job duration and operational requirements for both.