Retrieval-Augmented Generation (RAG)

RAG, or Retrieval-Augmented Generation, is an AI framework that enhances the accuracy and relevance of large language model (LLM) responses by integrating information retrieval from external knowledge sources before generating text.

RAG enabled you to be able to provide always-updated data to your foundation model without retraining it

• LLMs are combined with knowledge bases or document stores.
• Instead of cramming everything into the prompt, relevant context is retrieved in real-time and injected into the prompt dynamically.
  I initially had the impression that it might be simply appending data to a prompt sting or simply finding and replacing (and that is a part of the process). The process of retrieving the data can be sophisticated by using vector search or similarity checks.

A REST API that fetches user data from a DB and inserts it into a prompt that is then passed to an LLM can be considered a basic or simplistic form of RAG. But it lacks the sophistication of semantic search and relies on exact match/structured query

RAG search is a hybrid approach that:

• retrieves relevant documents or data using vector search (semantic similarity)
• augments the prompt sent to the LLM with those documents
• generates a final answer that's grounded in context

RAG Pipeline (Simplified)

1. Preprocess Data
   • Split documents into chunks (e.g., 500 words)
   • Generate embeddings for each chunk
   • Store embeddings in a vector database
2. At Query Time
   • Embed the user’s question
   • Use vector search to retrieve top-k similar chunks
   • Inject those into the prompt: Based on the following documents: [doc1], [doc2], ... Answer: "How do I register for sales tax in Quebec?"
3. Send to LLM and get answer

✅ Why RAG is Useful