Retrieval-Augmented Generation (RAG) pairs a search system with a large language model so answers are grounded in your data, not just what the model memorized during training. Watch the animation above — Maya's question travels to a retriever, which searches a vector database, hands back the right sentence, and the LLM turns it into a real answer: "You have 30 days from your purchase date."
Code Example
Three lines do the real work: retrieve the relevant sentence, assemble it into a prompt, and let the LLM generate a grounded answer.
Perplexity answers questions by retrieving live web pages and citing them inline — the same retrieve-then-generate pattern Maya's question just used, applied to the open web.
Retraining a model on new policy docs costs time and money and is stale the moment it finishes. RAG lets ByteToAI update the refund policy and have Maya's next question answered correctly instantly.
Each of those three file types needs a different extraction tool before it becomes plain text the rest of the pipeline can use.
Code Example
Each loader returns plain text, but extraction quality differs — a layout-aware PDF parser keeps the refund policy's sentences intact instead of scrambling them.
LlamaIndex ships 150+ "readers" that ingest Slack threads, Notion pages, S3 buckets, and scanned PDFs into a normalized Document format any RAG pipeline can consume.
Layout-aware parsers preserve table structure, so a later question like "what does the Team plan cost" pulls the right cell instead of a scrambled row of numbers.
Raw extracted text is messy: repeated page headers and footers, broken character encodings, duplicate paragraphs, and leftover navigation boilerplate from scraped HTML.
Code Example
One regex removes the repeated footer completely, leaving only the two sentences that actually matter for Maya's question.
Regulated deployments run PII scrubbers such as Microsoft Presidio over documents before indexing, so patient or client names never enter a vector store that might be queried broadly.
HIPAA and GDPR require sensitive data to be redacted before storage — cleaning isn't just a quality step here, it's a legal gate the pipeline must pass.
LLMs and embedding models have limited context, so documents get split into chunks small enough to embed and retrieve individually, but large enough to keep meaning intact.
Code Example
Splitting on sentence boundaries instead of a hard character count is what keeps chunk A a complete, answerable sentence.
Most production RAG stacks start with a recursive splitter that tries paragraph, then sentence, then word boundaries before falling back to a hard character cut.
A little overlap between chunks prevents a sentence that answers the question from being cut exactly at a boundary, without duplicating so much that the index bloats.
An embedding model converts each chunk into a vector of numbers such that semantically similar chunks land close together in space — meaning becomes geometry.
Code Example
Watch the canvas — particles carrying the real chunk text drift toward chunks with similar meaning, exactly like nearby vectors in a real embedding space.
Both use domain-tuned embedding models — a code-aware model for GitHub, a document-aware model for Notion — rather than a generic off-the-shelf embedding.
Switching from a generic to a domain-tuned embedding model can improve retrieval recall by 20–30% without touching the LLM at all.
Vector databases store embeddings and find the "nearest" ones to a query in milliseconds using an approximate nearest neighbor (ANN) index instead of comparing against every single vector.
Code Example
The search returns chunks A, B, C — the shelf near Maya's question — without ever touching chunk D.
Many teams (Notion, Shopify Sidekick) add pgvector to a Postgres database they already run, avoiding a new distributed system just for vectors.
Reusing Postgres cuts operational overhead at the cost of raw ANN throughput compared to a purpose-built vector database like Pinecone or Qdrant at huge scale.
Dense retrieval (embeddings) captures meaning; sparse retrieval (BM25/keyword) captures exact terms like IDs and product names; hybrid blends both into one score.
top_k and never the fusion weight α between dense and sparse scores.Code Example
Drag α — at 1.0 you get pure semantic (dense) ranking, at 0.0 pure keyword (BM25) ranking, and in between a fused score that usually performs best.
Both ship built-in hybrid search combining BM25 with vector similarity via reciprocal rank fusion, so teams don't have to hand-roll the fusion logic.
Hybrid retrieval reliably boosts recall@10 by double digits over dense-only search on standard benchmarks like BEIR.
The retriever's job is recall — get plausible candidates fast. The reranker's job is precision: a slower, more accurate model re-scores each candidate against the query and reorders them.
Code Example
The cross-encoder confirms chunk A — "within 30 days of purchase" — is the strongest match for Maya's exact question.
Teams bolt a hosted reranker like Cohere Rerank onto an existing retriever with a single API call, no model hosting required.
Teams commonly report 10–15% answer-accuracy gains just from adding a reranker on top of an otherwise unchanged retriever.
Guardrails are checks placed around the RAG pipeline to stop bad inputs, leaked data, and ungrounded answers from ever reaching the user.
Code Example
The output guard checks that "30 days" in the final answer really does come from chunk A, not from the model's imagination.
Enterprise RAG deployments wrap the whole pipeline in a guardrail layer like these, checking input, retrieval scope, and output before a response ever reaches the user.
Regulated industries like finance and healthcare require an auditable guardrail layer to pass compliance review before a RAG assistant can ship.
A production RAG system keeps improving after launch by capturing feedback — thumbs up/down, follow-up questions — and running automated evaluation to find weak spots in retrieval or generation.
Code Example
Every lap in the animation is one feedback cycle: an answer goes out, feedback comes back, gets logged, evaluated, and feeds an improvement back into the pipeline.
Teams pair RAGAS metrics with observability tools like LangSmith or Arize Phoenix to monitor production RAG quality continuously, not just at launch.
A 5% drop in context precision after a policy-doc refresh flags a chunking regression automatically, before it shows up as a wave of "Maya"s complaining.
The refund policy gets read, cleaned, and chunked once, ahead of time. From here on, every stage happens live, in under two seconds, the moment Maya hits enter.
Code Example
Every function call in this snippet maps to a stage on the timeline above — step through it to see where the two seconds actually go.
Internal copilots and answer engines like Perplexity trace this exact multi-stage journey per request, timing every hop from query to cited answer.
End-to-end tracing (e.g. via LangSmith) lets teams pinpoint whether a bad answer came from retrieval, reranking, or generation in seconds instead of hours of guesswork.
You've now covered the full RAG pipeline: multi-modal ingestion, cleaning, chunking, embedding, vector storage, hybrid retrieval, reranking, guardrails, and feedback loops — the same architecture behind production systems like Perplexity, enterprise copilots, and internal document search assistants.
Starter Stack
This four-line stack (splitter + vector store + retrieval chain) is a real, runnable starting point for everything you just learned.
Both frameworks ship end-to-end RAG quickstarts; RAGAS docs cover setting up automated evaluation once your pipeline is live.
Take three sentences from a document you actually have, chunk them, embed them, and ask a question — you'll see the exact same nine stages play out.