Retrieval-Augmented Generation gives an agent access to knowledge beyond its training data and beyond the context window — a vector database is the retrieval half of that pattern.
Core concepts
- Embeddings: A vector representation of text (or images) such that semantically similar content ends up numerically close together — the foundation everything else in this chapter builds on. Generate them with your model provider (OpenAI Embeddings, Voyage, Cohere Embed) or open models via Hugging Face.
- RAG pipeline basics: Chunk your source documents, embed them, store the vectors, then retrieve the closest matches to a query and feed them into the model's context. See also our vector search using vector embeddings walkthrough.
- Chunking strategy: How you split documents (by size, by section, by semantic boundary) has an outsized effect on retrieval quality — the most commonly underrated step in a RAG pipeline.
- Hybrid search: Combining vector similarity with traditional keyword search catches cases pure embedding similarity misses, especially exact terms like product codes or names.
What Is It?
Vector databases store embeddings for similarity search powering RAG.
Pipelines chunk documents, embed, index, then retrieve at query time.
Hybrid search mixes keyword and vector scores for better recall.
Why Should I Care?
Agents without retrieval hallucinate on proprietary facts.
Chunking and metadata filtering determine answer quality more than model choice.
Real World Analogy
Embeddings turn text into map coordinates — vector search finds the closest landmarks to your question GPS pin.
Visual Mental Model
ingest -> chunk -> embed -> index
question -> embed -> top-k -> promptBeginner Explanation
RAG feeds retrieved chunks into the prompt. This hub covers embeddings, chunking, hybrid search, and vendor options in child pages.
Smallest Possible Example
chunks = ["doc part 1", "doc part 2"]
# embed -> store -> query top-k at ask time
print("RAG pipeline", len(chunks))Common Mistakes
❌ Giant chunks that dilute relevance.
Retrieve paragraphs of noise; answers get vague.
❌ No metadata filters (tenant, doc type).
Cross-customer leakage via search results.
❌ Skipping eval questions for retrieval.
You optimize blind — measure hit rate on real queries.
Real Project Example
Notion Q&A, Glean, and Microsoft Copilot enterprise search use RAG pipelines.
Startups prototype RAG on pgvector before graduating to Pinecone.
Build This Mini Project
Chunk a markdown README into 500-token pieces manually; write five quiz questions and note which chunk should answer each.
Interview Question
What is RAG in one sentence?
Retrieve relevant documents via embeddings (often with filters), inject them into the prompt, then generate an answer grounded in that context.
Key Takeaways
- Hub orients RAG + vector DB options.
- Chunking and metadata matter immensely.
- Evaluate retrieval, not just generation.
- Hybrid search often beats pure vectors.
Resources
YouTube learning
Core concepts from video walkthroughs:
The 60-second story
A librarian who only knows titles will miss the book you need when you ask for “the one about starting over,” and the title says nothing like that. Embeddings are a way to shelve by meaning. A paragraph becomes a list of numbers. Similar meanings land nearby. The librarian can walk to the right aisle without an exact keyword.
RAG is the employee who is not allowed to answer from vibes. You chunk the company documents, embed the chunks, store them, and when a question shows up you fetch the nearest chunks and set them on the desk before the model speaks. The model is smart. It is not psychic about your refund policy.
Chunking is how you tear the books. Tear only by page count and you split a rule in half. Tear on headings and the rule stays intact. This is the step people rush, then they blame the model for a policy it only saw the bottom half of.
Hybrid search is the librarian who checks both the meaning aisle and the card catalog. Product codes, error strings, and names are exact. Pure “nearby meanings” can stroll past ERR-441 because nothing else in the aisle looks like it. Keyword plus vector is how you catch both the poem and the serial number.
Map the library.
- An embedding is a shelf position by meaning.
- A chunk is a torn page you hope is still a whole thought.
- The vector store is the building.
- Retrieval is walking to the nearest shelves.
- The prompt is the desk where you place those pages.
- Hybrid search is meaning plus the card catalog.
chunks = split(docs)
index(embed(chunks))
pages = search(embed(question))
answer = model(question + pages)Four beats. If pages are wrong, a better model writes a better wrong answer. Fix the shelves.
RAG exists because context windows cannot hold the company, and because training or fine-tuning is the wrong tool for a policy that changed Thursday. Retrieval is the Thursday update. You re-index. You do not re-train the employee’s personality.
Mental model: a librarian who shelves by meaning, and still keeps a card catalog for exact names. Put the pages on the desk. Then ask the question.
You have the page if you blame a bad chunk before you blame the model, and if you know why a SKU needs a keyword. Meaning gets you close. The card catalog keeps a product code from being shrugged off as poetry.
