Free tool · No signup
RAG Cost Calculator
Estimate what your RAG system costs to index once and to run each month, from your own documents, traffic and models.
Start from a scenario in the guide
Monthly running cost
$73.02
$876.18 a year
Cost per question
$0.018
Model only: $0.012
Questions per month
4,000
16 million tokens in, 1.6 million out
One-time indexing
$0.14
7.22 million tokens to embed
Tokens per question
4,000
Prompt, plus 400 answer
Where the money goes
- Answer generation$48/mo · 66%
- Hosting$25/mo · 34%
- Embeddings (re-indexing and queries)$0.02/mo · 0%
- Re-ranking$0/mo · 0%
- Vector store$0/mo · 0%
Answer generation is 66% of the total, as it usually is. Fewer or shorter chunks, or a smaller model for simple questions, cut it fastest.
These are running costs. Want a fixed quote for building it?
Building the system (ingestion, permissions, citations, evaluation and the interface) is quoted separately for a fixed price after a free scoping call, and isn't calculated here.
Your scenario goes into the message so you don't have to type it again. You can edit it before sending.
Every assumption
- Content
- 2,000 documents × 5 pages = 10,000 pages, 500 words a page
- Tokens per word
- 1.3 (650 tokens a page). Other languages often use more.
- Chunking
- 500-token chunks with 50 tokens of overlap: 14,445 chunks
- Changed each month
- 5% of documents, embedded again
- Questions
- 20 users × 10 a day × 20 days = 4,000 a month
- Prompt per question
- 900 instructions and history + 100 question + 6 chunks × 500 = 4,000 tokens
- Answer
- 400 tokens
- Model calls per question
- 1
- Generation model
- Claude Sonnet 5.5 ($2 in / $10 out per 1M tokens)
- Embedding model
- OpenAI text-embedding-3-small ($0.02 per 1M tokens, 1,536 dimensions)
- Re-ranking
- Off
- Vector store
- $0 a month
- Hosting
- $25 a month
How it's calculated
- Tokens = pages × words per page × tokens per word
- Chunks = tokens ÷ (chunk size − overlap). Tokens embedded = tokens × chunk size ÷ (chunk size − overlap)
- Prompt per question = instructions + question + top-k × chunk size
- Generation = questions × calls × (prompt × input price + answer × output price) ÷ 1M
- Embeddings per month = (changed % × tokens embedded + questions × question tokens) × embedding price ÷ 1M
- Re-ranking = questions × candidates × (question + chunk size) × price ÷ 1M, or questions × searches × price ÷ 1,000
- Total = generation + embeddings + re-ranking + vector store + hosting
Build cost and running cost
A RAG (retrieval-augmented generation) system costs money twice. Building it is paid once: getting your documents in cleanly, enforcing who can see what, adding citations and proving the answers are right. Running it is paid every month to your AI and hosting providers. This calculator covers the running cost. The build depends on your sources, permissions and interface, so it's quoted separately; How much does a custom RAG system cost explains what moves that number, and uses the same three scenarios as the buttons above.
What drives each line
- One-time indexing. Every document is split into chunks and each chunk is embedded. The overlap between chunks is embedded twice, which is why 50 tokens of overlap on 500-token chunks adds about 11%. Even a large library usually costs a few dollars to embed.
- Answer generation. Each question sends your instructions, the question and the top-k retrieved chunks to the model, which then writes the answer. The chunks are most of the prompt, and output tokens cost several times more than input.
- Embeddings each month. Documents that change are embedded again, and every question is embedded once to search with. Both are small unless your library changes heavily.
- Re-ranking. Optional: fetch 20 to 50 candidates, then let a reranker keep the best few. It adds a small cost per question, and often pays for itself by letting you send fewer chunks to the model.
- Vector store and hosting. Fixed monthly costs that don't change with traffic: the API server, the database and any sync jobs that keep the index up to date.
Why generation usually dominates
Take the internal docs scenario: 4,000 questions a month, each reading about 4,000 tokens of instructions and chunks. That's 16 million input tokens a month, against a one-time 7 million to embed the whole library. Generation is paid on every question; embedding mostly once. So the biggest savings come from the prompt: retrieve fewer, better chunks, trim chat history, and route simple questions to a smaller model. Switching the generation model in the calculator usually moves the total more than anything else.
When the vector database starts to matter
For most teams it's a small line. pgvector in a PostgreSQL database you already run is free, and handles hundreds of thousands to a few million chunks well. The calculator shows your index size under the vector store. A managed vector database starts to earn its fee with very large indexes, heavy query traffic or strict latency targets, or when you'd rather not run the database yourself. At low traffic, the fixed costs (vector store and hosting) can outweigh the model cost, which is why the cost per question falls as usage grows.
What the estimate doesn't include
- Prompt caching and batch discounts. Prices are standard rates. Caching the fixed part of the prompt can cut input costs noticeably on some providers.
- Free tiers and credits, such as Voyage's free first 200 million tokens.
- People time: someone keeping the documents current and reviewing questions the system got wrong.
- The build itself. For that, see what custom RAG development includes, or a website support chatbot for a single public site.
Model prices change often. They were checked on the date shown under the model pickers; recheck your provider's pricing page before you budget.
By Muhammad Saif · Updated
Questions, answered.
Is RAG cheaper than fine-tuning?
Usually, for answering questions from your own documents. RAG needs no training run, and updating it means re-embedding the changed files, which costs cents. Fine-tuning teaches a model a style or format well but is poor at keeping facts current, and every content change means another training run. Many teams fine-tune only when they need a specific output format, and use RAG for the knowledge.
Can I run it with self-hosted models?
Yes. Pick Custom price, set the per-token prices to 0 and put the GPU servers in hosting. GPU servers are billed by the hour whether or not anyone asks a question, so they're often the largest line item at low volume. They make sense when text can't leave your servers, or when volume is high enough that the fixed cost beats per-token prices.
How often should documents be re-indexed?
Only when they change. A sync job that re-embeds new and edited files, and deletes removed ones, keeps the index current at a fraction of the cost of rebuilding it. Re-embed everything only when you change the embedding model or the chunking, because vectors from different models can't be mixed.
Why does answer generation cost the most?
Each question sends the retrieved chunks to the model, so a question with 6 chunks of 500 tokens reads about 3,000 tokens before it writes a word, and output tokens cost several times more than input. Embedding is cheap by comparison and mostly paid once. Retrieving fewer, better chunks, re-ranking and routing simple questions to a smaller model are the biggest savings.
When does the vector database start to matter?
Later than most people expect. pgvector in a PostgreSQL database you already run handles hundreds of thousands to a few million chunks well. A managed vector database becomes worth paying for with very large indexes, heavy query traffic, or when you'd rather not run the database yourself. The calculator shows your index size to help you judge.
Is anything I enter stored?
No. The calculator runs entirely in your browser. Your numbers are kept in this browser and in the share link, and never sent to a server.