RAG in a nutshell

Retrieval-Augmented Generation (RAG) is a hybrid architecture that enhances large language models (LLMs) by combining parametric knowledge (stored in the model’s weights) with non-parametric external knowledge retrieved in real time. Instead of relying solely on the LLM’s pre-trained parameters, RAG first encodes a user query into a vector embedding, performs a similarity search (typically via cosine similarity or approximate nearest neighbours) against a vector database containing chunked and embedded documents, retrieves the most relevant passages (top-k), and injects them into the LLM’s prompt context. This grounding step significantly reduces hallucinations, enables access to up-to-date or proprietary information, and improves factual accuracy without the need for expensive fine-tuning. The typical pipeline involves an embedding model (e.g., text-embedding-3-small), a vector store (e.g., FAISS, Pinecone, Chroma), and a generative LLM (e.g., GPT-4o or Llama 3).

Key Technical Use Cases

1. Private Document Question-Answering (Enterprise Knowledge Base)

Organisations index internal PDFs, wikis, or SharePoint documents. A query is routed through a retriever to pull relevant chunks, then passed to the LLM for synthesis.

from langchain_community.vectorstores import Chroma

from langchain_openai import OpenAIEmbeddings, ChatOpenAI

from langchain_core.prompts import ChatPromptTemplate

from langchain_core.runnables import RunnablePassthrough

from langchain_core.output_parsers import StrOutputParser

# Vector store setup (run once)

embeddings = OpenAIEmbeddings()

vectorstore = Chroma.from_documents(docs, embeddings) # docs = list of Document objects

retriever = vectorstore.as_retriever(search_kwargs={"k": 6})

# RAG chain

llm = ChatOpenAI(model="gpt-4o-mini")

template = """Answer the question using only the provided context:

Context: {context}

Question: {question}

Answer:"""

prompt = ChatPromptTemplate.from_template(template)

def format_docs(docs):

  return "\n\n".join(doc.page_content for doc in docs)

rag_chain = (

  {"context": retriever | format_docs, "question": RunnablePassthrough()}

  | prompt

  | llm

  | StrOutputParser()

)

response = rag_chain.invoke("What is our policy on remote working?")

2.Agentic Workflows with Tool-Augmented RAG

In multi-step agents (e.g., your n8n setups), RAG acts as a memory or knowledge tool. The agent decides when to retrieve before calling other tools.

# Example LangChain agent tool

from langchain.tools import tool

@tool

def rag_knowledge_base(query: str) -> str:

  """Retrieve and synthesise information from the company knowledge base."""

  return rag_chain.invoke(query) # reuse chain above

3. Conversational Memory + RAG (Chat with Session History)

Combine conversation history (last N turns) with document retrieval for context-aware, personalised responses — ideal for your AI Training Academy coaching bots.

from langchain_core.messages import AIMessage, HumanMessage

# Add chat history to the chain

contextualize_prompt = ChatPromptTemplate.from_messages([

  ("system", "Given chat history, rephrase the follow-up question..."),

  MessagesPlaceholder("chat_history"),

  ("human", "{question}")

])

# Full chain can then feed contextualised question + retrieved docs

4. Real-time Data / DeFi or Compliance Use Cases

Index blockchain events, regulatory docs, or market feeds. Retrieve latest protocol parameters before generating investment summaries or compliance checks.

RAG is particularly powerful in production because retrieval is cheap, scalable, and can be cached or re-ranked (e.g., with Cohere Rerank or LLM-as-judge). For advanced setups, consider advanced chunking strategies (semantic, hierarchical), metadata filtering, or hybrid search (BM25 + vector). This architecture underpins many of the agentic systems you build with n8n and LangChain.