Knowledge bases with RAG: letting agents answer from your own documents

The agent that made up a policy. Picture Sara, a shop owner who runs a small online skincare shop. At eleven at night a customer messages her shop: can I still get a refund after ten days? Sara's AI agent replies in seconds, sounding confident: yes, you have thirty days. But Sara's real policy is fourteen days. Now she either honours a promise she never made or she upsets a customer. The model wasn't lying on purpose. It simply never saw her policy, so it filled the gap with a typical answer from the internet. Today we fix that by letting the agent read Sara's own documents before it replies.

What you will learn. By the end of this lesson you'll be able to do three things. First, explain what RAG is and why it makes an agent stop guessing. Second, describe how your business documents get split into small pieces and turned into something a computer can search by meaning. Third, wire the whole thing together on your own private server, using n8n and a vector database running in Docker. No maths background is needed, and you'll see a tiny working example in Python along the way.

RAG is an open-book exam. RAG stands for retrieval-augmented generation. That's a mouthful, so think of an open-book exam. A student who studied years ago might misremember a detail. Give them the right book and they look it up before writing. RAG does the same for your agent. Retrieval means: before answering, search your documents and pull out the few passages that relate to the question. Generation means: the language model writes the reply using those passages as its notes. You never retrain the model. When your policy changes, you just change the book on the desk.

Step one: prepare the book. Before your agent can look anything up, you prepare the book once. Gather the documents customers actually ask about: your FAQ, price list, refund and booking policies, even good past email replies. Then split them into chunks, which are small pieces of a few paragraphs each. Why chunks? Because you want to hand the model the one paragraph about refunds, not your whole forty-page handbook. Each chunk is then turned into an embedding, which we'll explain next, and saved in a vector database. This preparation step is called indexing, and you rerun it whenever a document changes.

Embeddings: an address for meaning. An embedding is a list of numbers that an embedding model produces for a piece of text. Think of it as a shelf address in a library. Books about refunds sit together on one shelf, books about shipping on another. Texts with similar meaning get addresses close together. When a customer asks, can I get my money back, the question gets its own address, and it lands right next to the refunds shelf, even though it never uses the word refund. So how does the search work? It simply finds the chunks whose addresses are closest to the question's address, and hands those to the model.

A tiny retrieval in Python. Here's the retrieval step in miniature. You'll never write this code yourself, because n8n does it for you, but it shows what happens inside. At the top are four chunks from Sara's shop, and below them one customer question. To keep it runnable anywhere, we use a simple word-counting method called TF-IDF as a stand-in for embeddings. Then cosine similarity gives each chunk a score from zero to one for how closely it matches the question. Look at the output: the refund chunk scores highest and the others score zero. A real embedding model does the same job but matches meaning, not just words, so it would also catch money back. The top chunks are what we send to the model.

The answering flow on your server. Now let's see the full answering flow running on your own server. A customer sends a refund question through your website chat. Your n8n workflow passes the question to an embedding model, which returns its vector. The workflow searches Qdrant with that vector and gets back the top three chunks. Then it sends those chunks plus the question to the chat model, with one firm instruction: answer only from these notes, and if the answer isn't there, say you don't know and offer a human. That single sentence is what stops the agent from guessing.

Qdrant: the vector database. For the vector database, a good default is Qdrant. It's open source, it installs as a single Docker container on the same server where n8n runs, and your documents never leave your server. If you use Postgres, a common database, there's a shortcut: the pgvector extension adds vector search right inside that same database, so you run one less service. Either way, n8n ships with vector store nodes for both, so building the indexing workflow and the answering workflow is mostly dragging nodes and pasting connection details.

Example: a hair salon's knowledge base. Let's apply this to a client, a small hair salon that wants an assistant for bookings. Start with the price list on the wall: each service and its price becomes its own chunk, so a question about colour treatment finds the colour line. Next, the policy binder on the front desk: cancellation rules, deposits, opening hours, each section a chunk. Finally, the phone, where customers ask things like, can I move my Friday booking? The agent retrieves the cancellation chunk, answers with the real twenty-four-hour rule, and hands over to the booking workflow.

Common mistake: a stale book. Here's the mistake that bites almost everyone. The salon raises its prices and someone edits the price list document. The old chunks are still sitting in the vector database, and simply indexing the file again usually adds new chunks next to them, so the agent can keep quoting last year's prices with full confidence. The fix is to replace, not add. Save the source file name and date with each chunk. Then an n8n workflow can delete that file's old chunks and index the new version, whenever the file changes or simply every night. The source name also shows you where an answer came from.

Recap. Let's pull it together. RAG is an open-book exam for your agent: it searches your documents first, then answers from what it found. Chunks and embeddings are what make search by meaning work, so a question about money back finds your refund policy. And the system stays trustworthy only if the index stays fresh and the agent is allowed to say I don't know instead of guessing. With this in place, your agents answer from the business's real facts, on a server you control.

Your exercise. Before the next lesson, build the raw material for your first knowledge base. Pick one real business, your own or a client's. Write down ten questions customers genuinely ask, from messages or emails. Then gather the documents that answer them and split those documents into chunks of a few paragraphs. For each question, note which chunk holds the answer. If a question has no matching chunk, you've just found a gap in the documents, and that's valuable too. Next time we'll keep this server healthy with backups, updates and monitoring.