The RAG pipeline that survives contact with production

Retrieval-augmented generation is easy to demo and hard to run. The demo works because the corpus is small, static, and hand-picked by whoever is presenting. Production is none of those things: the corpus changes through the day, nobody is curating it, and the person asking is a customer with zero patience for a confident wrong answer.

Three things the demo skips

First, freshness. If your source of truth moves and your index doesn’t, you’re serving stale answers at full confidence. Treat index freshness as an SLA with monitoring behind it, not a nightly cron job that fails without anyone noticing.

Second, evaluation. Without an eval set, you can’t say whether a prompt change, a model upgrade, or a new chunking strategy helped or hurt — you’re guessing. Build the eval harness before the third feature, not after the first incident.

Third, a gateway. Every call to a model should go through one governed entry point handling routing, redaction, caching, rate limits, and cost attribution. Put it in early; adding it later across a dozen call sites is its own project.

The payoff

None of this shows up in the demo, and none of it is glamorous. It’s the whole difference between an AI feature that earns trust over time and one that slowly loses it.