Technical debt is familiar: shortcuts taken to ship faster that cost more later. Production AI adds a second kind. Call it reliability debt: the evaluation, monitoring and fallbacks that were skipped to get a model live.
It rarely shows up on launch day. It shows up weeks later, when inputs drift, a supplier changes a document format, or a model update shifts behaviour in a way no one noticed.
Where reliability debt hides
- No evaluation set. Without a fixed set of real cases to test against, every change is a guess.
- No production monitoring. Accuracy, latency and cost are tracked in the pilot, then forgotten.
- No fallback. When the model is unsure, there is no route to a person.
- No owner. The data science team built it; operations runs it; nobody owns its quality.
Paying it down
Build the evaluation set before the model. Collect real examples from the process, including the awkward ones, and agree what a correct answer looks like. Run every change against it.
Then monitor in production the same things you measured in the pilot, and set alerts on them. Decide in advance what happens when confidence is low: a review queue, a simpler rule, or a hand-off to a person.
Reliability is not a phase after launch. It is part of the definition of done.
Teams that budget for this from the start ship a little later and spend far less time firefighting. The ones that do not tend to find out what their AI really costs six months in.