Cost scenario · Chat
Customer support chatbot
A support assistant embedded in a product. It has a system prompt describing tone, escalation rules and what it must never promise, a handful of retrieved help-centre articles, and the conversation so far.
A conversation runs six turns before it resolves or escalates. The interesting part is that turn six costs several times turn one, because every turn resends everything before it.
The shape of one conversation
A 1,200 token system prompt plus 2,500 tokens of retrieved articles, both constant, against a transcript that grows each turn. Summed across six turns that is roughly 14,000 input and 1,800 output tokens for the whole conversation.
What it costs per month
Computed from the verified rates in the price table, last reviewed August 5, 2026. Figures assume caching is enabled — the uncached column shows what skipping that flag costs.
| Volume | Ministral 3 8BCheapest that fits | Magistral MediumMid tier | Claude Fable 5Frontier |
|---|---|---|---|
| Small product1,000 conversations / month | $2.37$2.37 uncached | $37.00$37.00 uncached | $160.70$230.00 uncached |
| Growing SaaS50,000 conversations / month | $118.50$118.50 uncached | $1,850.00$1,850.00 uncached | $8,035.00$11,500.00 uncached |
| Large deployment500,000 conversations / month | $1,185.00$1,185.00 uncached | $18,500.00$18,500.00 uncached | $80,350.00$115,000.00 uncached |
On this shape the spread between the cheapest model that fits and the frontier one is 97.0×, which is the largest single decision on this page. Enabling prompt caching removes a further 30% on Claude Fable 5, because 55% of the input on this workload is a stable prefix.
Where a provider publishes no cached input rate, the cached and uncached figures above are identical. That is a gap in what the provider discloses, not a model that cannot cache.
What actually drives the bill
The transcript resent on every turn
This is the dominant cost and the one nobody budgets for. A six-turn conversation does not send its content once — turn two carries turn one, turn three carries both, and so on. Cost grows with the square of conversation length, so doubling the average turns roughly quadruples the bill.
The constant prefix
System prompt plus retrieved articles is around 3,700 tokens that never change within a conversation and rarely change between them. Uncached, you pay full input rate for that on every single turn. This is the largest saving available and it is a configuration flag.
Retrieval breadth
Fetching eight articles instead of three roughly doubles the context. Retrieval quality is what lets you fetch fewer, so a better reranker is a cost control as much as an accuracy improvement.
How to cut it, in order of return
- Enable prompt cachingroughly 40% off the total
The system prompt and retrieved articles are a stable prefix. At roughly a tenth of base input rate for cached reads, and with more than half the input being that prefix, this is the single largest lever and it changes nothing about the product.
- Route easy turns to a smaller modelup to 60% on the routed share
Greetings, order-status lookups and "did you get my email" do not need a frontier model. Classify the turn first with something cheap, escalate only what is genuinely open-ended.
- Compact the transcript past turn four25–35% on long conversations
Replace the early turns with a short state summary once the conversation gets long. It flattens the quadratic curve, and long conversations are where the money is.
- Cap the answer length10–20% of output
Output costs several times input. "Answer in at most four sentences unless the customer asked for steps" is a cost instruction that also improves the product.
What people get wrong budgeting this
- Budgets built from a single-turn test are wrong by roughly the average turn count squared. Measure a whole conversation, not a message.
- Escalated conversations cost more than resolved ones, because they run longer before giving up. If a third of your traffic escalates, weight the average accordingly.
- Retries after a timeout are billed. A flaky retrieval step that causes 5% retries adds 5% to the bill invisibly.
Price your own version
The shape above is representative, not measured from your system. Put your real prompt and volume through these and the number becomes yours.
- Chatbot Conversation Cost CalculatorHistory is resent every turn, so cost grows quadratically. Most budgets miss this entirely.
- LLM API Cost CalculatorRequests per month in, dollars out. Input, cached input and output priced separately.
- Prompt Cache Structure CheckerFinds cache-busting content and measures the prefix that survives it.
- LLM Token CounterReal BPE tokenization, not characters ÷ 4. Shows which counts are exact and which are estimates.
Before committing to volume, the cost control checklist is ordered by saving per hour of effort, and every rate used here links to the provider page it came from on the Mistral AI and other provider pages.