Research-backed calculator guide

RAG Cost Calculator: Estimate Retrieval, Embeddings and Generation

Learn how to estimate RAG cost per query and per month by separating document embeddings, query embeddings, vector retrieval, reranking, generation, retries and operational allowances.

monthly RAG cost = ingestion embeddings
                 + query embeddings
                 + retrieval and storage
                 + reranking
                 + generation tokens
                 + retries and operations

Your RAG cost calculator is designed for planning when some of those meters are not yet available. It lets you enter requests, model steps, input and output tokens, cached-input share, retry rate, observability, human review, tools and a separate RAG & storage / task allowance. That allowance can represent embeddings, vector reads, storage and retrieval services until you can replace it with a usage export.

The result is a monthly estimate, cost per request, cost per successful task and a planning range. Use the dated rate card or enter custom rates before making a budget decision; it is an estimate, not a provider quote.

What the calculator measures

1. Generation cost

Generation is the model work after your application has assembled the prompt. In RAG, the input usually contains the user question, system instructions, conversation history, retrieved passages, tool results and formatting instructions. The output is the answer, structured response or next action.

The calculator models input tokens, cached input tokens, output tokens and the number of agent steps. This matters because a RAG workflow may call a model to rewrite a question, classify a request, retrieve context, assess evidence and then draft an answer. If each task has four model steps, a prompt-cost estimate based on one call understates usage by roughly four times before retries are considered.

The provider bills the token meters according to its own rate card. For example, the current OpenAI pricing record used by the site distinguishes input, cached input and output rates, and describes standard and batch processing separately. Check the provider page because these terms and prices can change. ^[2]

2. Document embeddings

Before a knowledge base can support semantic retrieval, its content is split into chunks and converted into vectors. The embedding request is generally priced by input tokens, although the exact meter depends on the provider. OpenAI describes an embedding as a vector of floating-point numbers and says embedding requests are billed by the number of input tokens. ^[1]

Document embedding is often a one-time ingestion cost, but it becomes recurring when files change, content is re-indexed, or chunking and model choices are tested. Keep it separate from query traffic. A large corpus can create a small initial API charge but a meaningful storage bill, while frequent re-indexing can make ingestion a recurring line item.

3. Query embeddings

A semantic-search query normally needs its own embedding. Its token count is usually much smaller than the indexed corpus, but it occurs on every retrieval request unless the application caches or reuses the result. The recurring query-embedding formula is:

query embedding cost
= monthly retrieval queries
× average query tokens
÷ 1,000,000
× embedding input rate

If one user request performs two independent searches, count two query embeddings. If a hybrid search uses both keyword and vector retrieval, the keyword path may have a different cost profile, but it still consumes application and database resources.

4. Retrieval, storage and reranking

Vector retrieval includes the database or search service that stores vectors and returns nearest matches. Its bill may depend on stored vector size, reads, writes, namespaces, replicas, query volume, returned fields, egress or a minimum plan. A managed vector provider may expose read and write units rather than a simple per-query price. Pinecone’s public pricing, for example, lists storage, write units, read units and egress as separate database dimensions and states that examples exclude inference and initial import. ^[3]

Reranking is another possible meter. Your system may first retrieve 20 candidates and then use a reranker to select the best five. Count the reranker request or unit separately when the provider does so. If you do not yet have that data, include a conservative allowance inside RAG & storage / task and document what it covers.

The RAG cost formula

For a granular model, calculate ingestion and query costs separately:

corpus embedding cost
= corpus tokens ÷ 1,000,000 × embedding rate

monthly query embedding cost
= monthly retrieval queries × query tokens ÷ 1,000,000 × embedding rate

monthly retrieval cost
= vector reads + writes + storage + egress + reranking

Then calculate generation for the model calls:

model cost per step
= (new input tokens × input rate)
+ (cached input tokens × cached-input rate)
+ (output tokens × output rate)

base generation cost
= monthly requests × steps per task × model cost per step

Finally add per-task operational allowances and retries:

base operating cost
= base generation cost
+ tool calls
+ RAG and storage
+ observability
+ human review

retry premium
= base operating cost × retry rate

monthly total
= base operating cost + retry premium

This is the same planning shape used by the calculator. It does not assume that every vector database or embedding API has the same unit. Instead, you can convert known usage into a per-task allowance, or enter a custom rate when your vendor exposes a more precise meter. The methodology page explains why list prices, estimates and invoices should remain distinct.

Worked example: 10,000 RAG requests per month

Use the calculator’s visible default scenario as a reproducible example, not a universal benchmark. It assumes:

• 10,000 requests per month and a 92% successful-task rate.
• Four model steps per task.
• 2,400 input tokens per step and 650 output tokens per step.
• Twenty percent of input tokens treated as cached input.
• An 8% retry allowance.
• $0.012 per task for tools and web calls.
• $0.006 per task for RAG and storage.
• $0.004 per task for observability and no human-review allowance.
• The site’s dated OpenAI GPT-6 Luna record: $0.10 per million new input tokens, $0.01 per million cached input tokens and $0.50 per million output tokens. This is a dated public list-price assumption, not a quote. ^[2]

The model cost per step is:

(2,400 × 80% × $0.10 / 1,000,000)
+ (2,400 × 20% × $0.01 / 1,000,000)
+ (650 × $0.50 / 1,000,000)
= $0.0005218 per step

Four steps cost $0.0020872 per task. Across 10,000 tasks, base model-token cost is $20.87. Tools and RAG add $180 ($0.018 × 10,000), and observability adds $40. The pre-retry subtotal is therefore $240.87.

The 8% retry allowance adds $19.27, producing an estimated monthly total of $260.14. Dividing by 10,000 requests gives about $0.026 per request. Dividing by 9,200 successful tasks gives about $0.028 per successful task. Cost per successful task exposes failed retrievals and repair loops.

The example is not a universal answer. The $0.006 RAG allowance is an editable planning input, not a universal price for embeddings or a specific vector database. To make the estimate more precise, replace it with a measured monthly equivalent:

RAG allowance per task
= (monthly query embeddings
 + retrieval reads
 + reranking
 + storage allocation
 + re-indexing allocation)
÷ monthly requests

For quarterly re-embedding, divide the next re-indexing bill by three, then add recurring query and retrieval costs. Keep ingestion visible so migrations do not look like operational drift.

Assumptions and exclusions

The calculator treats each request as a task with one or more model steps and uses representative token averages. Measure real traces when possible, including instructions, history, retrieved chunks, tool output and the final response.

The RAG allowance can cover embeddings, vector reads, storage, egress, reranking or a managed retrieval API. Do not add a charge twice; remove embedding inference if it is already included in the invoice.

The estimate does not automatically know your chunking, vector dimensions, replicas, plan minimum, region, tax, discount, network transfer, multimodal usage or self-hosted infrastructure. It also cannot infer hidden usage or every rounding rule. A free tier or monthly minimum can dominate a small workload even when per-query arithmetic looks tiny.

For token assumptions, use the AI Token Cost Calculator; for a model-and-workflow view, use the LLM Cost Calculator.

Common mistakes when estimating RAG cost

Counting only the answer tokens. Retrieved passages are input tokens. If the same context is sent through several model steps, it can become the largest model meter.

Treating ingestion as free after the first upload. Re-indexing, changed documents, chunk-size experiments and embedding-model migrations create new embedding requests and vector writes.

Using document size as token count. A megabyte, page or character count is only a proxy. Tokenize representative files. OpenAI’s embedding documentation points to tiktoken for counting tokens before an embedding request. ^[1]

Calling every search one request. A single user question may produce query rewriting, multiple namespace searches, a rerank call and a second retrieval pass. Count each billable or resource-consuming operation.

Ignoring retries and low success rates. A cheap request that succeeds only after two attempts has a different unit economics story. Report cost per successful task, not only cost per HTTP request.

Comparing rates without matching units. One provider may bill per million tokens, another per read unit, another by storage and plan minimum. Normalize the workload first, then compare the resulting monthly and per-successful-task cost.

Practical optimization and reconciliation steps

Start with retrieval quality, not an arbitrary smaller chunk. Test chunk size and overlap against recall, answer quality and the number of chunks passed to the model. More chunks can increase vector reads and generation input without improving the answer.

Cache safely repeatable work. Query embeddings for identical normalized questions, stable system context and unchanged document embeddings may be reusable, subject to your provider’s rules and freshness requirements. The AI Token Cost Calculator is useful for testing the effect of a different cached-input share, but treat that share as measured behavior rather than a guaranteed discount.

Use metadata filters and scoped indexes when they improve relevance and reduce candidate work. Apply reranking only where it changes quality enough to justify its meter. Set maximum retrieval and repair limits; an unbounded retry loop turns a rare failure into an unpredictable bill.

After launch, reconcile the estimate with the same month’s usage export. Match request counts first, then compare document-embedding tokens, query-embedding tokens, vector reads and writes, reranker calls, model input and output tokens, retries and any plan minimum. If the gap remains, check region, tax, discounts, credits and billing-period cutoffs separately. The Observability Cost Calculator can help you budget the traces and evaluations needed to explain that gap.

FAQ

Is RAG cheaper than sending the whole document to the model?

Not automatically. Retrieval can reduce the context sent to the generator, but it adds embeddings, indexing, vector search and sometimes reranking. Compare complete cost per successful answer and quality, not only prompt length.

What is RAG cost per query?

RAG cost per query is the variable cost of query embedding, retrieval, reranking and generation for one request. Add a monthly share of ingestion and storage when you want a fully loaded unit cost. If one user request triggers multiple searches or model steps, include them all.

Do embeddings cost money every time a user asks a question?

Document embeddings are usually created during ingestion or re-indexing. Query embeddings may be created for every semantic-search request. Separate those two volumes so a one-time corpus operation is not confused with recurring traffic.

Can the calculator price my vector database exactly?

It can model a transparent per-task RAG and storage allowance, which is useful before you have detailed telemetry. Exact billing requires your provider’s storage, read, write, egress, reranking and plan-minimum terms. Enter a custom allowance based on that rate card and verify it against an invoice.

Which input should I optimize first?

Start with the largest cost driver in the result. Often that is repeated model context, high retrieval allowances, or retries rather than the embedding API itself. Change one assumption at a time and check quality, latency and cost per successful task.

Related tools

Use the RAG Cost Calculator for a complete retrieval-and-generation scenario. Pair it with the LLM Cost Calculator when you are comparing model steps, and the Observability Cost Calculator when traces, evaluations and retention are material. The Methodology page documents rate records and calculation boundaries.

About the Author

Zunair Ali Syed is a doctoral researcher in Management Sciences at Minhaj University Lahore and a research consultant with experience in academic research and editing since 2011. He runs QuickEduHelp, with professional interests in digital learning, research methods, and evidence-based academic support.