Evaluating & Maintaining Quality for Data Agents

Swap a model, tweak a prompt, or change the context an agent pulls in, and an answer that was right last week can be wrong today, with nothing in your logs to flag it. Chris Collins shows how evals catch these regressions: what a useful eval includes, how to run evals nightly and in CI, and how to tell a real regression from a flaky run. Using DataHub's eval system as the example, he also covers how to test a context change before it ships.