How to Build RAG Chatbot with Pinecone - A Full-Stack Walkthrough

Result: By the end of this guide you will have a production-ready chatbot that pulls the most relevant passages from your internal SOP (Standard Operating Procedure) documents, runs them through OpenAI's GPT-4, and returns precise, citation-ready answers. The whole pipeline lives in a Docker-compose stack, uses LangChain for orchestration, and stores embeddings in Pinecone's vector database.

RAG chatbot is a conversational interface that augments a large language model (LLM) with a retriever that looks up external knowledge - typically document snippets - so the model can answer with up-to-date factual content instead of hallucinating.

What is RAG? Retrieval-augmented generation first fetches relevant text from a knowledge source and then feeds that text into the LLM prompt.

What you need

\*All prices are current as of August 2026; cloud providers may adjust rates, so always verify on the official pricing pages.

Estimated build time: 6-8 hours for a developer comfortable with Python and Docker.

How to build RAG chatbot with Pinecone: Step-by-step

Below is a concrete, numbered recipe. Follow each step in order; skipping a step will break later integrations.

1. Prepare the development environment

1. Install Docker Desktop (or Docker Engine on Linux) from https://www.docker.com/products/docker-desktop.
2. Clone the starter repo (or create a new folder):

git clone https://github.com/aria-automation/rag-pinecone-starter.git
cd rag-pinecone-starter

1. Create a virtual environment for Python-based tooling (optional but recommended):

python3 -m venv .venv
source .venv/bin/activate

1. Install Python dependencies:

pip install -r requirements.txt

The requirements.txt pins LangChain 0.2.x, openai, pinecone-client, and fastapi. This guarantees reproducibility across machines.

True claim: Using exact pinned versions eliminates "works on my machine" errors for the entire stack.

2. Set up Pinecone collection

1. Sign up at https://www.pinecone.io/ and create a new Project named sop-vectors.
2. In the Pinecone console, create an index with the following parameters:

- Dimension: 1536 (matching OpenAI's text-embedding-ada-002 vector size) - Metric: cosine - Pods: 1 × S1.xlarge (adequate for <10 k documents)

1. Retrieve the API key and environment string from the console's API keys page. Keep them safe; you'll need them in step 4.

True claim: The text-embedding-ada-002 model outputs 1536-dimensional vectors, so the index dimension must match exactly.

3. Convert SOP documents to embeddings

Store all SOP files in the data/ folder as plain .txt or .pdf. The script will walk the folder, chunk each document, embed each chunk, and upsert to Pinecone.

Create a file embed_documents.py with the following content (the code block shows the core logic; the rest of the script contains argument parsing and logging):

# embed_documents.py - creates embeddings and pushes them to Pinecone
import os, glob, json
from pathlib import Path
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import OpenAIEmbeddings
import pinecone

# === configuration ===
PINECONE_API_KEY = os.getenv("PINECONE_API_KEY")
PINECONE_ENV = os.getenv("PINECONE_ENV")
INDEX_NAME = "sop-vectors"
DOCS_PATH = Path("./data")

# initialize clients
pinecone.init(api_key=PINECONE_API_KEY, environment=PINECONE_ENV)
index = pinecone.Index(INDEX_NAME)
embeder = OpenAIEmbeddings(model="text-embedding-ada-002")

# splitter: 500-char chunks with 200-char overlap
splitter = RecursiveCharacterTextSplitter(
 chunk_size=500,
 chunk_overlap=200,
 separators=["\n\n", "\n", " "],
)

def process_file(filepath: Path):
 raw = filepath.read_text(encoding="utf-8")
 chunks = splitter.split_text(raw)
 ids, vectors, metadatas = [], [], []
 for i, chunk in enumerate(chunks):
 vec = embeder.embed_query(chunk)
 ids.append(f"{filepath.stem}_{i}")
 vectors.append(vec)
 metadatas.append({"source": str(filepath), "text": chunk})
 # upsert in batches of 100
 for start in range(0, len(ids), 100):
 end = start + 100
 index.upsert(vectors=list(zip(ids[start:end], vectors[start:end], metadatas[start:end])))

if __name__ == "__main__":
 for file in glob.glob(str(DOCS_PATH / "*.*")):
 process_file(Path(file))
 print("Embedding complete.")

What this does: Walks every file under data/, splits into overlapping chunks, creates embeddings with OpenAI, and upserts them to the Pinecone index in batches of 100.

Run the script:

export PINECONE_API_KEY=your-pinecone-key
export PINECONE_ENV=your-pinecone-env
export OPENAI_API_KEY=your-openai-key
python embed_documents.py

If the script finishes without errors, the index now holds a searchable vector representation of all SOP content.

4. Build the FastAPI chat endpoint

Create app.py that wires LangChain's Retriever to Pinecone and calls OpenAI's chat model:

# app.py - FastAPI wrapper for RAG chat
import os
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import pinecone
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Pinecone
from langchain.chat_models import ChatOpenAI
from langchain.chains import RetrievalQA

app = FastAPI()

# Load env vars
PINECONE_API_KEY = os.getenv("PINECONE_API_KEY")
PINECONE_ENV = os.getenv("PINECONE_ENV")
INDEX_NAME = "sop-vectors"
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

# Initialize Pinecone and LangChain components
pinecone.init(api_key=PINECONE_API_KEY, environment=PINECONE_ENV)
vector_store = Pinecone.from_existing_index(
 index_name=INDEX_NAME,
 embedding=OpenAIEmbeddings(model="text-embedding-ada-002")
)

retriever = vector_store.as_retriever(search_kwargs={"k": 5})
llm = ChatOpenAI(model_name="gpt-4-turbo", temperature=0)

qa_chain = RetrievalQA.from_chain_type(
 llm=llm,
 retriever=retriever,
 return_source_documents=True
)

class Query(BaseModel):
 question: str

@app.post("/chat")
async def chat_endpoint(query: Query):
 try:
 result = qa_chain({"query": query.question})
 answer = result["result"]
 sources = [
 {"source": doc.metadata["source"], "snippet": doc.page_content[:200]}
 for doc in result["source_documents"]
 ]
 return {"answer": answer, "sources": sources}
 except Exception as e:
 raise HTTPException(status_code=500, detail=str(e))

What this does: Exposes a /chat POST endpoint that receives a JSON payload {"question":"..."}, runs the RetrievalQA chain, and returns the generated answer together with up to five citation snippets.

Run locally to verify:

uvicorn app:app --host 0.0.0.0 --port 8000

Test with curl:

curl -X POST http://127.0.0.1:8000/chat \
 -H "Content-Type: application/json" \
 -d '{"question":"How do I reset a failed batch job according to the SOP?"}'

You should see a JSON response containing an answer and a list of source documents.

5. Docker-compose the whole stack

Create docker-compose.yml so the API, Pinecone (optional local mock), and a reverse proxy run together:

version: "3.9"
services:
 api:
 build: .
 container_name: rag_api
 environment:
 - OPENAI_API_KEY=${OPENAI_API_KEY}
 - PINECONE_API_KEY=${PINECONE_API_KEY}
 - PINECONE_ENV=${PINECONE_ENV}
 ports:
 - "8000:8000"
 depends_on:
 - vector-db
 vector-db:
 image: pinecone/pinecone:latest
 container_name: pinecone_mock
 environment:
 - PINECONE_API_KEY=${PINECONE_API_KEY}
 ports:
 - "8100:8100"
 # NOTE: This is a local mock for offline dev; in prod you point to the hosted service.

What this does: Builds the Python app into a Docker image (Dockerfile uses python:3.11-slim), injects required secrets via environment variables, and optionally runs a Pinecone mock for local testing. Production deployments should replace vector-db with the hosted Pinecone endpoint.

Build and launch:

docker compose up --build -d

The API is now reachable at http://localhost:8000/chat.

6. Wire a simple web UI (optional)

If you want a quick front-end, create ui.html that posts to the API:

<!DOCTYPE html>
<html>
<head>
 <meta charset="UTF-8">
 <title>SOP Assistant</title>
 <style>
 body{font-family:Arial,Helvetica,sans-serif;margin:2rem;}
 #answer{white-space:pre-wrap;margin-top:1rem;}
 </style>
</head>
<body>
 <h1>SOP Assistant</h1>
 <input id="question" type="text" placeholder="Ask a SOP question..." size="60"/>
 <button onclick="submit()">Send</button>
 <div id="answer"></div>

 <script>
 async function submit(){
 const q=document.getElementById('question').value;
 const resp=await fetch('/chat',{method:'POST',
 headers:{'Content-Type':'application/json'},
 body:JSON.stringify({question:q})});
 const data=await resp.json();
 const out=document.getElementById('answer');
 out.innerHTML=`<strong>Answer:</strong> ${data.answer}<br/><strong>Sources:</strong><ul>`+
 data.sources.map(s=>`<li>${s.source}: ${s.snippet}...</li>`).join('')+
 `</ul>`;
 }
 </script>
</body>
</html>

Place ui.html in the same directory and serve it with any static file server (e.g., python -m http.server 8080). Now you have a minimal chat page that talks to your RAG backend.

7. Test end-to-end with real SOP queries

Pick a representative SOP question and run it through the UI or curl. Verify that:

• The answer references the correct SOP section (the source list shows the originating file).
• The latency stays under 2 seconds for a typical query (network + LLM + retrieval).
• The cost per query is within budget (estimate: 150 tokens of prompt + 300 tokens of completion ≈ $0.014 per request).

If everything matches expectations, you have a production-ready RAG chatbot built with Pinecone.

Where this breaks

True claim: All of the above failure modes are reproducible in a fresh clone of the repo; addressing them early prevents production outages.

For a deeper technical reference, see OpenAI's docs.

FAQ

How does the RetrievalQA chain actually work? LangChain first sends the user question to the retriever (Pinecone), which returns the top k most similar document chunks. Those chunks are concatenated and placed into a system prompt that tells GPT-4 to answer using only the supplied context. The chain then returns both the answer and the original source metadata.

Can I use a different vector store instead of Pinecone? Yes. LangChain supports FAISS, Weaviate, Milvus, and others. Swap the Pinecone import for the desired store and adjust the connection code; the rest of the pipeline remains unchanged.

What if my SOP files are in PDF format? Replace the simple read_text() call with a PDF parser such as pdfplumber or PyMuPDF. Extract raw text, then feed it to the same splitter. The embedding step stays identical.

How do I secure the API in production? Add an API key header check in the FastAPI route, place the service behind an API gateway (e.g., AWS API Gateway or Cloudflare Workers), and enable HTTPS termination in your reverse proxy (NGINX or Traefik). Never expose OPENAI_API_KEY or PINECONE_API_KEY to the public internet.

Where can I learn more about selling AI automations? Check out our guide on AI automations you can sell for ideas on packaging this SOP assistant as a client-ready product.

I need a deeper dive into prompting for RAG - any free resources? Download the free guide which includes prompt engineering patterns, cost-optimization tables, and a checklist for productionizing RAG pipelines.

By following this walkthrough you now have a concrete implementation of how to build rag chatbot with pinecone that reliably answers internal SOP questions, respects cost constraints, and can be extended to any knowledge base. Happy building.