TLDR: What Matters in Production RAG

Date: 2026-05-15 Source: https://arpitbhayani.me/blogs/rag-production

Overview

Most of us build RAG the same way: follow a tutorial that embeds a handful of PDFs, stores the vectors in a local Chroma instance, and chains everything together with LangChain (if that's still a thing). The answer looks reasonable. Then you take it to production and it falls apart in quiet, hard-to-diagnose ways.

Key Points

• Most of us build RAG the same way: follow a tutorial that embeds a handful of PDFs, stores the vectors in a local Chroma instance, and chains everything together with LangChain (if that’s still a thing).
• RAG Basics: The core idea is simple: instead of asking an LLM to answer from memory, you retrieve relevant documents at query time and inject them into the prompt as context.
• RAG Indexing Pipelines: Here is where most tutorials stop and most production problems begin.
• Observability and Retrieval Tracing: Production RAG systems fail in ways that look like LLM problems but are actually retrieval problems.