what is rag ai: a no-PhD guide to retrieval-augmented generation

RAG is a technique that combines a large language model (LLM) with a vector store of embedded text chunks to retrieve relevant context at query time. In practice you feed a user prompt to the LLM, first pull the most relevant passages from an external knowledge base, then let the model generate a response that is grounded in those passages. This matters because it dramatically reduces hallucinations, lets you keep the model's knowledge up-to-date without costly fine-tuning, and lets you answer domain-specific questions with a single API call.

What you need

Estimated build time: 2-3 hours for a minimal proof-of-concept, 1-2 days for a production-ready pipeline with monitoring.

Step-by-step build

1. Prepare your document corpus - Place all text files (PDFs, markdown, HTML) in a folder ./docs. - Use a simple Python script (or n8n "Read Binary File" node) to extract raw text.

2. Chunk the text - Split each document into ~200-token chunks to stay within the LLM's context window. - In n8n, add a Function node with JavaScript:

 // This node receives `content` as a string and returns an array of chunks
 const maxTokens = 200;
 const words = $json["content"].split(/\s+/);
 const chunks = [];
 for (let i = 0; i < words.length; i += maxTokens) {
 chunks.push(words.slice(i, i + maxTokens).join(' '));
 }
 return [{ json: { chunks } }];
 

3. Create embeddings for each chunk - Add an HTTP Request node that calls OpenAI's embedding endpoint.

 {
 "method": "POST",
 "url": "https://api.openai.com/v1/embeddings",
 "headers": {
 "Authorization": "Bearer {{ $env.OPENAI_API_KEY }}",
 "Content-Type": "application/json"
 },
 "body": {
 "model": "text-embedding-ada-002",
 "input": "{{$json.chunk}}"
 },
 "responseFormat": "json"
 }
 

What this does: Sends each 200-token chunk to OpenAI and receives a 1536-dimensional vector.

4. Store embeddings in Pinecone - Use another HTTP Request node pointing to Pinecone's upsert endpoint (https://{index}.svc.{region}.pinecone.io/vectors/upsert). - Payload example:

 {
 "vectors": [
 {
 "id": "doc-{{ $json.docId }}-{{ $json.chunkIdx }}",
 "values": {{ $json.response.data[0].embedding }},
 "metadata": { "text": "{{ $json.chunk }}" }
 }
 ]
 }
 

5. Build the retrieval-augmented query workflow - Trigger: HTTP Webhook node (exposes /query). - Step A - Embed the user query: Same HTTP Request node as in step 3, but with the user's prompt as input. - Step B - Retrieve top-k similar chunks: HTTP Request to Pinecone's query endpoint, requesting topK=5.

 {
 "method": "POST",
 "url": "https://{index}.svc.{region}.pinecone.io/query",
 "headers": {
 "Authorization": "Bearer {{ $env.PINECONE_API_KEY }}",
 "Content-Type": "application/json"
 },
 "body": {
 "vector": {{ $json.response.data[0].embedding }},
 "topK": 5,
 "includeMetadata": true
 },
 "responseFormat": "json"
 }
 

What this does: Finds the five most relevant document chunks for the user's question.

6. Compose the final prompt for the LLM - Use a Set node to concatenate the retrieved texts into a system prompt:

 You are an assistant that answers using only the provided context. Context:
 {{ $json.results.map(r => r.metadata.text).join('\n---\n') }}

 Question: {{ $json.question }}
 

7. Call the LLM - Add an HTTP Request node to OpenAI's chat completion endpoint (https://api.openai.com/v1/chat/completions).

 {
 "method": "POST",
 "url": "https://api.openai.com/v1/chat/completions",
 "headers": {
 "Authorization": "Bearer {{ $env.OPENAI_API_KEY }}",
 "Content-Type": "application/json"
 },
 "body": {
 "model": "gpt-4",
 "messages": [
 { "role": "system", "content": "{{ $json.composedPrompt }}" }
 ],
 "temperature": 0.2,
 "max_tokens": 500
 },
 "responseFormat": "json"
 }
 

8. Return the answer - Connect the response to the Webhook node's return JSON:

 {
 "answer": "{{ $json.choices[0].message.content }}",
 "sources": {{ $json.results.map(r => r.id) }}
 }
 

9. Deploy and test - Run the workflow locally (docker compose up -d n8n) or on n8n Cloud. - POST a JSON payload to https://your-n8n-instance.com/webhook/query with {"question":"What is rag ai?"}. - Verify that the answer cites the retrieved chunks (the sources array).

Result: You now have a live endpoint that answers "what is rag ai" (or any domain question) by grounding the response in your own knowledge base, dramatically reducing hallucinations.

Where this breaks

Key truth: RAG does not eliminate hallucinations outright, but it cuts them by roughly 30-40 % when the retrieved context is high-quality (see OpenAI's best-practice guide).

For a deeper technical reference, see n8n's documentation.