Cost scenario · Retrieval
RAG over your documentation
A question-answering layer over a documentation set — around 5,000 pages, chunked into roughly 40,000 pieces. Users ask a question, the system retrieves the five most relevant chunks and asks a model to answer from them.
Two costs that are usually budgeted separately and should not be: the one-off indexing, and the per-query cost that retrieval multiplies by fetching context.
The shape of one query
A 30 token question, a 600 token system prompt, and five retrieved chunks of about 800 tokens each. The system prompt is the only cacheable part, because retrieved chunks differ per query.
What it costs per month
Computed from the verified rates in the price table, last reviewed August 5, 2026. Figures assume caching is enabled — the uncached column shows what skipping that flag costs.
| Volume | Ministral 3 8BCheapest that fits | Magistral MediumMid tier | Claude Fable 5Frontier |
|---|---|---|---|
| Internal tool5,000 querys / month | $3.75$3.75 uncached | $56.00$56.00 uncached | $288.60$330.00 uncached |
| Public docs site100,000 querys / month | $75.00$75.00 uncached | $1,120.00$1,120.00 uncached | $5,772.00$6,600.00 uncached |
| High traffic1,000,000 querys / month | $750.00$750.00 uncached | $11,200.00$11,200.00 uncached | $57,720.00$66,000.00 uncached |
On this shape the spread between the cheapest model that fits and the frontier one is 88.0×, which is the largest single decision on this page. Enabling prompt caching removes a further 13% on Claude Fable 5, because 20% of the input on this workload is a stable prefix.
Where a provider publishes no cached input rate, the cached and uncached figures above are identical. That is a gap in what the provider discloses, not a model that cannot cache.
What actually drives the bill
Retrieval multiplies input by 150×
A 30 token question becomes a 4,600 token request. Everything about RAG cost follows from that one fact: you are not paying for questions, you are paying for the context you attach to them.
Chunks retrieved, not chunks stored
Indexing 40,000 chunks is a one-off embedding cost measured in single-digit dollars. Retrieving five of them on every query, forever, is the recurring one. Storage is cheap; context is not.
Cacheability is low here
Unlike a chatbot, most of a RAG request is different every time. Only the system prompt caches, so the usual "just enable caching" advice moves this workload much less than it moves others.
Re-embedding on document change
Documentation changes. Every edited page is a re-embed, and switching embedding model means re-embedding the entire corpus with no incremental path.
How to cut it, in order of return
- Retrieve three chunks instead of fiveabout 35% of input
Test it before assuming quality drops. Past a point the relevant chunk competes with irrelevant ones, so fewer and better often improves the answer as well as the bill.
- Add a reranker30–40%, indirectly
A reranker lets you retrieve three good chunks instead of eight hopeful ones. The extra model call costs far less than the context it removes.
- Cache the answers, not just the prompt20–40% on public docs
Documentation questions repeat heavily. Your own cache on the question plus retrieved-set hash costs nothing and eliminates the request entirely.
- Route to a cheaper model for extractive answersup to 80% of the routed share
When the answer is present verbatim in the retrieved chunk, a small model reproduces it as well as a large one. Synthesis across chunks is where the expensive tier earns its price.
What people get wrong budgeting this
- Embedding cost gets quoted as the cost of RAG. It is usually under one percent of the first year; the queries are the system.
- Chunk overlap inflates both storage and retrieval. Fifty percent overlap on 40,000 chunks means paying to store and retrieve the same text twice.
- A failed retrieval still costs the full request if the model answers anyway. Define what happens when nothing relevant comes back.
Price your own version
The shape above is representative, not measured from your system. Put your real prompt and volume through these and the number becomes yours.
- Embedding Cost CalculatorIndex build, re-indexing and query embedding, priced separately.
- Vector Database Storage EstimatorStorage by dimensions and precision, with quantisation savings shown.
- RAG Retrieval Budget CalculatorWhat retrieved context costs per month, and the k comparison table.
- Text Splitter for RAG and EmbeddingsToken-accurate chunking with configurable overlap. Respects sentence and paragraph boundaries.
Before committing to volume, the cost control checklist is ordered by saving per hour of effort, and every rate used here links to the provider page it came from on the Mistral AI and other provider pages.