The full retrieval-augmented generation pipeline in concrete terms, chunking, embeddings, vector database similarity search, and how retrieved chunks get injected into an actual client.messages.create() call, plus why RAG exists at all: grounding answers in a knowledge base that changes daily without retraining or fine-tuning the model.
ragvector-databaseembeddingsretrievalai
DeepFrontend Pro Topic
Unlock the Full Deep Dive on RAG Basics & Vector Databases
You are viewing the 40% free preview. Upgrade to DeepFrontend Pro to unlock the complete 30-minute masterclass, live interactive engines, and staff-level interview loops.
✓Under the Hood: Deep architectural internals & edge cases
✓Interview Q&A: Senior/Staff model answers & red flags
A model's training data is frozen at a point in time and has no idea about your company's internal documentation, this morning's product changelog, or a customer's specific account history. Retrieval-Augmented Generation (RAG) solves this without retraining or fine-tuning anything: instead of hoping the model already knows the answer, you retrieve a small, relevant slice of a much larger knowledge base at request time and hand it to the model as context, then ask it to answer using that context.
RAG has two entirely separate phases that run at different times: ingestion (offline, run once or on a schedule, whenever source documents change) and query-time retrieval (online, run on every single user question).
Phase 1, Ingestion: documents become searchable vectors#
// 40,000 characters, far too large for one embedding call
Raw document
active
40,000 chars
→
Chunker
idle
not yet run
RAG starts offline, before any user query exists. Raw source documents are collected, docs, PDFs, wiki pages, usually far too large to embed as a single unit.
import fs from "fs";const doc = fs.readFileSync("handbook.md", "utf-8"); // e.g. 40,000 characters// 1. Chunk the document, far too large to embed as a single unitconst chunks = splitIntoChunks(doc, { size: 500, overlap: 50 });// → ~85 overlapping chunks of ~500 characters each// 2. Embed each chunk, a SEPARATE model from the one that will generate answersconst vectors = await Promise.all( chunks.map((chunk) => embeddingModel.embed(chunk.text)),);// → each chunk.text becomes a fixed-length float vector (e.g. 1536 dimensions)// 3. Store vector + original text + metadata in a vector databaseawait vectorDb.upsert( chunks.map((chunk, i) => ({ id: chunk.id, vector: vectors[i], metadata: { source: chunk.source, text: chunk.text }, })),);
Chunking with overlap (overlap: 50 above) exists to prevent a fact from being silently split across a chunk boundary, without overlap, a sentence that happens to straddle exactly where one 500-character chunk ends and the next begins can become unretrievable, because neither half alone is similar enough to a relevant query to surface in search. The embedding model is deliberately a different model from the one that later generates answers, its only job is producing a numeric vector that captures a chunk's meaning, not generating text.
At query time, the user's question is embedded using the exact same embedding model used during ingestion, mixing embedding models between ingestion and query silently breaks similarity search.
// 1. Embed the user's question, with the SAME embedding model used at ingestionconst question = "What is our refund window for annual plans?";const queryVector = await embeddingModel.embed(question);// 2. Similarity search against the vector databaseconst results = await vectorDb.query({ vector: queryVector, topK: 4 });// → the 4 nearest stored vectors, ranked by similarity (commonly cosine distance)// 3. Build context from ONLY the retrieved chunks, not the whole knowledge baseconst context = results.map((r) => r.metadata.text).join("\n---\n");// 4. Generate a grounded answer using client.messages.create()const response =
This is the entire point of RAG: pull in a small, relevant slice of a much larger knowledge base for THIS specific question, instead of stuffing the entire knowledge base into every request (which would be both prohibitively expensive in tokens and, past a certain size, would simply exceed the context window). The final answer is grounded in the retrieved chunk's actual current wording rather than the model's frozen training-data memory, which is exactly what lets a RAG system stay current with a knowledge base that changes daily, with zero retraining.
The single most important invariant: same embedding model, both times#
Using the same embedding model at ingestion and at query time is not a minor detail, mixing embedding models silently breaks similarity search entirely, because different embedding models produce vectors in entirely different, mutually meaningless vector spaces. A query vector from model A compared against stored vectors from model B will not return relevant results, and, critically, this failure produces no error at all; it just quietly returns poor-quality or irrelevant chunks, which is a much harder bug to diagnose than an outright crash.
Instructing the model to answer only from context#
The system prompt in the query-time example above deliberately constrains Claude to the retrieved context. Without this instruction, the model may blend its own training-data knowledge with the retrieved context, which can reintroduce exactly the staleness problem RAG exists to solve, a well-designed RAG system prompt typically also tells the model what to say when the retrieved context doesn't actually answer the question, rather than letting it guess.