How to build a RAG pipeline.
A practical, step-by-step guide to building a retrieval-augmented generation pipeline that answers from your own documents, with a Python example you can read.
ZENKEI
ZENKEI
A practical, step-by-step guide to building a retrieval-augmented generation pipeline that answers from your own documents, with a Python example you can read.
To build a RAG pipeline, you ingest and chunk your documents, embed those chunks into a vector database, retrieve the most relevant ones for each question, and have a language model generate an answer grounded in them with citations. The steps below walk through the whole flow, from raw documents to a system that answers from your own knowledge.
RAG stands for retrieval-augmented generation. A RAG pipeline is the sequence of steps that lets a language model answer from your own documents instead of guessing from what it memorised in training. It retrieves the passages relevant to a question, then asks the model to answer using only those passages, with citations back to the source. That grounding is what makes the answers accurate and verifiable.
A RAG pipeline is a set of small, swappable parts rather than one big model. Before building, get these in place:
Gather the sources and clean them: strip boilerplate, extract text from PDFs, keep useful metadata like title, section and URL. Quality here decides quality everywhere downstream.
Split each document into passages of a few hundred tokens, with a little overlap so context is not cut mid-thought. Chunk size is the single biggest lever on retrieval quality, so it is worth tuning.
Run every chunk through the embedding model to get a vector. Chunks about the same topic end up near each other in vector space, which is what makes semantic search work.
Index the vectors, together with their text and metadata, in a vector database. This is what you query at answer time, and it is cheap to rebuild when documents change.
For each question, embed the query and pull the top matching chunks. Add re-ranking or hybrid keyword search when plain vector search misses. Good retrieval is where most of the quality comes from.
Put the retrieved chunks into the prompt and instruct the model to answer only from them, and to cite sources. If the context does not contain the answer, the model should say so rather than invent one.
Build a small evaluation set of questions and expected sources. Measure whether retrieval finds the right passages and whether answers are faithful, then tune chunking, retrieval and prompts.
Here is the core loop in a few lines: embed the chunks, retrieve the closest ones for a question, and ask a model to answer from them. Real systems add a vector database, re-ranking, citations and evaluation, but the shape is the same.
from sentence_transformers import SentenceTransformer
import numpy as np
embedder = SentenceTransformer("all-MiniLM-L6-v2")
# 1. chunk your documents, then embed and index them
chunks = chunk_documents(docs, size=500, overlap=50)
index = embedder.encode(chunks, normalize_embeddings=True)
def answer(question):
# 2. retrieve the top-k most relevant chunks
q = embedder.encode([question], normalize_embeddings=True)
scores = index @ q.T
top = np.argsort(scores.ravel())[-4:][::-1]
context = "\n\n".join(chunks[i] for i in top)
# 3. generate an answer grounded in the retrieved context
prompt = (
"Answer using ONLY the context. Cite sources. "
"If the answer is not in the context, say you do not know.\n\n"
f"Context:\n{context}\n\nQuestion: {question}"
)
return llm(prompt)Swap the in-memory index for a real vector database once you have more than a few thousand chunks, and add citations by carrying each chunk's source metadata through to the answer.
A prototype is a weekend; a reliable, private RAG system with good retrieval, citations and evaluation is real engineering. That is exactly our RAG development service. If you also need the model tuned to your domain, see custom LLM development, or explore the full AI hub.
A RAG pipeline is the sequence of steps that lets a language model answer from your own documents: ingest and chunk the documents, embed them into a vector database, retrieve the most relevant chunks for a question, and generate an answer grounded in those chunks with citations.
Collect and prepare documents, chunk them into passages, embed the chunks, store them in a vector database, retrieve the top matches for a query, generate a grounded answer with citations, and evaluate and iterate on quality.
Start with a strong general embedding model and a simple vector database such as pgvector or Qdrant. The right choice depends on your data volume, languages and privacy needs, and can be swapped without rewriting the whole pipeline.
Improve retrieval so the right passages are found, instruct the model to answer only from the retrieved context, show citations so answers are verifiable, and add evaluation to catch regressions. Most hallucinations in RAG come from weak retrieval, not the model.
Usually no. RAG gives an off-the-shelf model your knowledge at answer time, so most projects need no fine-tuning. Fine-tuning helps when you also need a specific tone, format or domain skill, and the two can be combined. See custom LLM development.
Bring your documents. We build a private RAG system that answers with citations.