How to Build a RAG Chatbot With n8n
This page may contain affiliate links. If you buy through them we may earn a commission at no extra cost to you.
Direct answer: a RAG (retrieval-augmented generation) chatbot in n8n needs two separate workflows: an ingestion pipeline that loads your documents, splits them into chunks, generates embeddings, and stores them in a vector database, and a chat workflow where an AI Agent node uses a Vector Store Tool to retrieve relevant chunks and answer questions grounded in them. Building a working version with a handful of test documents takes about an hour; tuning retrieval quality so it reliably answers real questions takes longer, and that tuning, not the n8n wiring, is where most of the real work is.
Step 1: Pick a vector store
n8n has dedicated nodes for Pinecone, Supabase (via pgvector), Qdrant, and a few others. If you don't already run a vector database, Supabase is a reasonable starting point since it's regular Postgres underneath and useful for other things too. Pinecone trades that flexibility for a fully managed service with less infrastructure to reason about. Qdrant is the better call if you want to self-host for data control; our guide to self-hosting n8n covers the equivalent self-hosting tradeoffs for the automation layer itself. Whichever you pick, create an empty index or table before moving to ingestion.
| Vector store | Type | Best for |
|---|---|---|
| Supabase (pgvector) | Managed Postgres | Already running Postgres, want one less service |
| Pinecone | Fully managed, vector-only | Minimal infrastructure, managed scaling |
| Qdrant | Self-hostable | Full control over data residency |
Step 2: Build the ingestion workflow
Start a new workflow with a trigger appropriate to your source, a manual trigger for a one-time import, a Schedule Trigger if documents update regularly. Add a Document Loader node pointed at your source (local files, Google Drive, a URL list), followed by a Text Splitter node that breaks each document into chunks.
Chunk size is the setting that matters most here, more than which embedding model or vector store you chose. Too large, and a chunk mixes unrelated information, diluting what gets retrieved. Too small, and a chunk loses the surrounding context that gave it meaning in the first place. We started at 500 tokens with 50 tokens of overlap for a support-documentation knowledge base and found that a reasonable middle ground, but the right number depends on how your source material is structured; a FAQ with short, self-contained answers tolerates smaller chunks than a technical manual with cross-referencing paragraphs.
Step 3: Generate embeddings and store them
Add an Embeddings node (OpenAI, Cohere, or another supported provider) between the splitter and your vector store node. Connect the vector store node (Pinecone Insert, Supabase Insert, or equivalent) and run the workflow once against your test documents. This writes every chunk, as a vector plus its original text, into your store.
Treat this as a repeatable workflow, not a one-time script. Source content changes, and a separate, ongoing version of this same pipeline, triggered by a schedule or a webhook when a document updates, is what keeps your chatbot's answers current. Skipping this is the most common reason a RAG chatbot's answers quietly go stale.
Step 4: Build the chat workflow with an Agent and a Vector Store Tool
Start a second workflow with a Chat Trigger for testing, or a Webhook for production use. Add an AI Agent node connected to a chat model, and attach a Vector Store Tool (pointed at the same store from Step 3) to the agent's tool input. This is the retrieval half: when a user asks a question, the agent calls the tool, which searches your vector store for the most relevant chunks and returns them as context before the LLM generates its answer.
Write the system prompt to constrain the agent to the retrieved context explicitly: tell it to answer only from the provided documents and to say plainly when it doesn't have enough information, rather than filling the gap with general knowledge. This single instruction did more to reduce confident-sounding wrong answers in our testing than any other prompt change. Our broader guide to building an AI agent in n8n covers the Agent node's other settings (memory, additional tools) that apply here too.
Step 5: Test retrieval quality, not just the final answer
Most RAG debugging time should go into checking what the Vector Store Tool actually retrieves, not just whether the final answer sounds right. Run a test query and inspect the tool's output in n8n's execution log directly; if the retrieved chunks aren't relevant to the question, no amount of prompt tuning on the agent side will fix it; the problem is upstream, in chunking or embedding quality.
Deliberately test questions your documents don't cover. A well-built RAG chatbot should say it doesn't know; one that confidently invents an answer to an out-of-scope question has a prompt that isn't constraining it tightly enough, or a retrieval step returning marginally-relevant chunks that the LLM treats as sufficient grounding when they aren't.
When NOT to build RAG in n8n
If you need enterprise-scale retrieval, millions of documents, sub-second latency requirements, or fine-grained access control per document, a dedicated RAG platform or a custom backend will handle that better than an n8n workflow calling a vector store node per request. n8n's strength here is fast iteration and integration with the rest of your automation stack, not raw retrieval throughput at scale. If your actual need is a general-purpose conversational agent rather than one grounded in your own documents, our guide to building an AI agent in n8n without the RAG layer is the simpler starting point. And if assembling the ingestion and chat pipelines from scratch isn't worth your time compared to a visual, less code-adjacent builder, our n8n vs OpenAI AgentKit comparison covers an alternative approach to the same agent-building problem.
For the messaging layer once your RAG chatbot works, our Slack bot with n8n guide and our Telegram bot with n8n guide cover wiring the same Agent node up to a chat interface your team or customers actually use.
Frequently asked questions
Which vector store should I use with n8n for a RAG chatbot?
Supabase (built on pgvector) is a reasonable default if you don't already run a dedicated vector database, since it doubles as regular Postgres for anything else you need. Pinecone is the more purpose-built, managed option with less infrastructure to think about, at the cost of a separate bill once you're past its free tier. Qdrant is the strongest pick if you want to self-host for data control. n8n has a dedicated node for all three, so the switching cost between them is lower than it looks.
Do I need to re-run the whole ingestion pipeline every time my source documents change?
Only for the documents that changed, if you design it that way. Most production setups split ingestion into its own workflow, triggered on a schedule or a webhook when a source file updates, rather than wiring it into the same workflow that answers chat questions. Treat ingestion and chat-answering as two separate, ongoing workflows from the start.
Why does my RAG chatbot give answers that aren't in my documents?
This is the LLM falling back on its own general knowledge when retrieval returns weak or irrelevant chunks, usually a chunking or embedding-model mismatch rather than a prompting problem. Tightening the system prompt to explicitly instruct the agent to say it doesn't know when the retrieved context doesn't answer the question reduces this significantly, though it won't eliminate it entirely.
How is this different from just building a regular AI agent in n8n?
A RAG chatbot is a specific kind of AI agent: one whose primary tool is a Vector Store Tool retrieving from your own documents, rather than an agent calling arbitrary APIs or taking actions. Our broader guide to building an AI agent in n8n covers the Agent node, memory, and tool-calling concepts this guide builds directly on top of.