Maintaining the evaluation program

Every technique in the evaluation half of this book produces an artifact — a dataset, a suite, a gate, a judge — and every one of those artifacts decays. A hidden set gets peeked at until it's contaminated; ground truth goes stale as the world changes; the input distribution drifts away from the test set; a judge's biases shift under a model update; the suite fills with redundant or obsolete cases. An evaluation program you don't maintain slowly stops measuring reality — while its dashboards stay reassuringly green. This final chapter is about treating evaluation as a living system with its own upkeep.

What you will understand by the end

• Why an evaluation program decays even when nothing "breaks."
• The main decay modes: contamination, stale ground truth, drift, judge rot, test bloat.
• How to keep the program healthy — refresh, re-audit, prune, re-calibrate.
• Why "evaluate your evaluation" is the habit that keeps every number trustworthy.

Evaluation decays silently

The insidious thing about evaluation decay is that it produces no error. A contaminated hidden set, stale labels, or a distribution that has drifted all yield a number that looks exactly as trustworthy as a healthy one — the process runs, the gate passes, the graph is green. But the number now describes a world that no longer exists, and the gap only shows up as unexplained production failures. Because nothing announces the rot, you have to go looking for it on a schedule.

The decay modes

• Contamination. The hidden set leaks through repeated peeking or training-data overlap; its number inflates. → Refresh over-exposed hidden sets.
• Stale ground truth. Correct answers that were right last year are wrong now (a changed policy, a new fact); the references themselves are outdated. → Re-audit ground truth periodically, especially where a strong system "fails."
• Distribution drift. Production shifts — new request types, phrasings, segments — and the representative set no longer mirrors it. → Re-sample from recent traffic and re-check coverage.
• Judge rot. An LLM judge's agreement and biases move when its model or prompt changes. → Re-calibrate after any judge change and on a schedule.
• Test bloat. The suite accumulates redundant, obsolete, or now-trivial cases that slow it down and dilute signal. → Prune and de-duplicate; retire tests that no longer earn their place.

Keep the program healthy

Maintenance is a recurring practice, not a one-off:

• Refresh hidden sets on a schedule and whenever peek-budget is spent, to fight contamination.
• Re-audit ground truth, prioritising cases a strong system fails — a signal of label noise.
• Re-sample and re-check coverage against recent production to keep the set representative as the distribution drifts.
• Re-calibrate judges after any model/prompt change and periodically regardless.
• Prune the suite — retire redundant, obsolete, and trivially-passing cases so it stays fast and meaningful.
• Track eval health as first-class metrics: hidden-set age, ground-truth audit date, coverage vs current traffic, judge agreement, suite runtime.

Mental model

Evaluation artifacts decay silently — contamination, stale ground truth, drift, judge rot, test bloat — producing confident numbers that describe a world that's moved on. The defence is to evaluate the evaluation on a schedule: refresh hidden sets, re-audit ground truth, re-sample for drift, re-calibrate judges, prune the suite, and track eval health as its own metrics.

Common mistakes

• Building the eval once and trusting it forever. It decays even when nothing visibly breaks.
• Never refreshing the hidden set. Repeated exposure contaminates it; the honest number quietly inflates.
• Assuming ground truth is permanent. Labels go stale; re-audit, especially where strong systems "fail."
• Letting the suite bloat. Redundant, obsolete tests slow it and dilute signal; prune regularly.

Practical guidance

• Put evaluation maintenance on a schedule: refresh hidden sets, re-audit ground truth, re-sample for drift, re-calibrate judges, prune the suite.
• Track eval-health metrics (hidden-set age, audit date, coverage vs traffic, judge agreement, runtime) and act on them like any other SLO.
• Prioritise re-auditing ground truth on cases a strong system fails — the fastest way to find label rot.
• Treat "design the next hidden test" as a standing task, so the evaluation evolves with the system it measures.

Summary

• An evaluation program is a living system that decays silently — the number stays confident while it stops describing reality.
• The decay modes: contamination, stale ground truth, drift, judge rot, test bloat — each with a maintenance answer.
• Keep it healthy by refreshing, re-auditing, re-sampling, re-calibrating, and pruning on a schedule, and tracking eval-health metrics.
• This project's "design the next hidden test" mindset is program maintenance made explicit — the habit that keeps every number worth trusting.

Knowledge check

A team's offline eval has said "94%, all green" for a year while production complaints climb. Nothing in the eval pipeline is broken. What's the most likely explanation?

The evaluation has decayed silently while still running fine. The most likely culprits: distribution drift (the test set no longer mirrors current production, so it measures an outdated input mix), stale ground truth (labels that were right a year ago are now wrong), or contamination (a hidden set peeked at for a year has inflated). All produce a confident, green number that describes a world that's moved on. The fix is to evaluate the evaluation: re-sample from recent traffic, re-audit ground truth, and refresh the hidden set — the maintenance that catches decay the pipeline can't report.

Why is a decayed evaluation more dangerous than an obviously broken one?

Because a broken evaluation announces itself — it errors, or produces obvious nonsense, so you fix it. A decayed one runs perfectly and produces a normal-looking, confident number that everyone trusts, while quietly measuring the wrong thing (a drifted distribution, stale labels, a contaminated set). So the team makes well-documented, confident, wrong decisions and blames the model when production disagrees. Invisibility is the danger: the only way to catch silent decay is to proactively audit the evaluation artifacts on a schedule, rather than waiting for the incident that finally reveals the rot.

Related chapters

• Holdout contamination — the decay mode maintenance refreshes away
• Drift detection — spotting the distribution shift that ages a test set
• Judge calibration — the re-calibration that fights judge rot
• Designing the next hidden test — program maintenance in the case study