problem_kicker
The hard part of production RAG is keeping retrieval correct, authorized, reproducible and measurable while sources, embeddings, prompts and models change. A useful architecture treats indexing, retrieval, ranking and generation as separately versioned systems.
DEMAND LANGUAGE / REAL-WORLD PROBLEM
“The RAG demo works. How do we make it reliable in production?”
“Vector DB + prompt was easy — how do we know the answers are actually getting better?”
WHAT CAUSES THIS?
Vector-only retrieval loses exact terms and identifiers.
architecture_for PRODUCTION RAG ARCHITECTURE
We define source-of-truth semantics, authorization boundaries and evaluation queries before selecting retrieval components. Index versions, embedding versions, prompt versions and model versions are explicit so a result can be replayed and compared.
ACL-aware retrieval should constrain candidates before protected evidence reaches generation. Service identities, index access and evaluation datasets should follow least privilege and preserve source authorization semantics.
Latency budgets are allocated per stage. Candidate count, fusion strategy, reranker depth, model context and cache policy are tuned against measured retrieval quality rather than optimized in isolation.
RAG · hybrid retrieval · reranking · CDC · embeddings · LLM observability
failure_kicker
measure_kicker
verify_intro
CTO / CIO FAQ
Use versioned indexes or versioned embedding fields, backfill incrementally, evaluate both paths and switch through an explicit compatibility gate.
For enterprise knowledge tasks, evidence links materially improve auditability. Citation correctness must still be evaluated; merely emitting a link is not enough.
Use a stable evaluation set and measure retrieval, evidence correctness, answer quality, latency and cost separately.
connected
