Evals are the CI/CD of your agents: a guide to checks that can actually fail

ELI25: what’s an eval?

Your unit tests check code: add(2, 3) returns 5, every time, forever. An LLM doesn’t work like that. Change a prompt, a model, or an agent’s operating rules and the output shifts in ways you can’t predict — and “it seems better” is a feeling, not evidence.

An eval is the test suite for that problem. You keep a fixed set of inputs (the golden set), a definition of what a good answer looks like (the rubric), and something that scores each output against the rubric — string checks and heuristics where you can, another LLM acting as judge where you must. Run the same set before and after any change and you get a number instead of a vibe: 92% before, 84% after means your “improvement” made things worse. Wire the number into CI with a threshold and a bad change can’t ship at all. That’s the whole idea, and it’s why people have started calling evals the CI/CD of agent engineering.

Capability was never my problem

I run a production agent stack solo: scheduled agents, capture pipelines, an LLM gateway with a per-key spend ledger. In August I audited a fortnight of fleet work and found something uncomfortable. Every significant defect traced back to verification, not capability. A check that couldn’t fail. An instrument answering a different question than the one I asked. An artefact confidently asserting something nobody had built. The models were fine. My ability to tell whether the models were fine was not.

The sharpest example is a gate I’d built myself. My pr-body-contract check (VAS-2229) blocks any PR whose body is missing its Problem, Outcome and Evidence sections. It works. It’s also honest about its own limitation, which I wrote into the ticket: it asserts presence, and “it cannot tell whether any of it is TRUE, so a green tick is never informative.” That’s the gap evals exist to close. A presence gate checks that the homework was handed in. An eval checks whether the answers are right.

Build your first eval in an afternoon

This is the pilot I scoped as VAS-1624, generalised into steps you can run on your own stack.

1. Pick your two highest-traffic prompts or skills. Traffic matters because regressions there cost the most and your incident history there is richest. Mine were linear-ops and session-close — the two skill files my agents load most often.

2. Mine your incident history for the golden set. Don’t invent test cases; excavate them. Every rule in a mature prompt or skill file exists because something went wrong. My linear-ops skill says “to mark a duplicate, pass duplicateOf alone — setting the state and the relation together fails.” I paid for that rule five separate times before encoding it. That’s a golden-set case: input “mark VAS-X as a duplicate of VAS-Y”, pass condition “emits duplicateOf only”. Ten to fifteen cases per skill is plenty to start.

3. Write the rubric as pass conditions, in order of preference: exact match, then heuristic, then LLM-as-judge. Deterministic checks are free and never drift. Save the judge model for genuinely qualitative criteria, give it a written rubric rather than “is this good?”, and spot-check its verdicts before you trust them.

4. Include at least one case that fails right now. This is the step people skip and the reason most suites are decorative. If every case passes on day one, you’ve proven the suite can’t fail, which is the exact defect you’re trying to eliminate. Find a known weakness — an edge your skill handles badly, a rule it states ambiguously — and encode it red. Watching it go green later is how you know the harness measures anything.

5. Wire it as a merge-blocking gate, not a dashboard. A score someone might look at is a gate that depends on a human noticing, and my dossier’s rule after a fortnight of counterexamples is blunt: no gate may depend on a human noticing. An alarm that lived as best-effort prose (VAS-2219), a runner four artefacts asserted but nobody provisioned (VAS-2220), a script that lost its executable bit and sat silently dead for two days (VAS-2191) — every one looked fine until it mattered. Block the merge or don’t bother.

6. Read the result back. A green run is not proof the run happened the way you think. I’ve had APIs return success while silently doing nothing, and a well-formed empty result is a fact about the instrument, not the world. Log which cases executed, assert the count, and treat “zero failures, zero cases” as a failure.

That last loop is the compounding part: every production incident becomes a permanent regression case, so the suite gets harder to fool as your stack gets older.

What this looks like at model-selection scale

The same method works above the prompt level. In July I ran an eight-model bake-off (VAS-1781) for my scheduled agents’ end-of-day workload: one rubric, one real workload, eight models scored side by side — Gemini Flash and Flash-Lite, Kimi, Qwen, DeepSeek, GLM, Grok, ChatGPT. Flash-Lite topped it, and I migrated the production crons on that number. Without the rubric that decision is a vibe about model brands; with it, it’s a measurement I can re-run when the next model drops. The bake-off predates my harness, which is precisely why VAS-1825’s acceptance criteria include re-running it through the harness as validation — an eval platform that can’t reproduce your best manual eval isn’t measuring what you think.

The judgement call: don’t build the platform

Here’s the decision that prompted this guide. I have a ticket for a proper reusable eval platform — versioned rubrics, experiment IDs, trend views, the lot (VAS-1825) — and a sibling ticket to decide whether Langfuse already is that platform (VAS-1175). Both are parked, on purpose, with the decision logged on the ticket.

The reason is a pattern in my own friction log: four recorded instances of infrastructure tails eating the product window. “Build the reusable platform” is exactly that shape. The two-hour pilot answers the question that actually matters — do evals catch real regressions in my stack? — and the platform question only deserves attention if the answer is yes. The parking comment carries an explicit unpark trigger: when the pilot’s go/no-go lands, the Langfuse adopt-vs-build decision is the first move, per the platform ticket’s own acceptance criteria.

If you’re advising a client, this is the shape to recommend: pilot first, platform on evidence, and write the unpark trigger down so “deferred” doesn’t quietly become “forgotten”.

Advice summary

Closing remarks, in the order I’d give them across a whiteboard:

• Audit your defects before your capabilities. If your last five agent incidents were verification failures, more capability won’t save you and better checking will.
• Excavate, don’t invent. Your incident history is a pre-paid golden set. Every rule in your prompts exists because something broke; turn each one into a regression case.
• One failing case, minimum. A suite that has never been red is a decoration. Prove the harness can fail before you trust its passes.
• Presence is not truth. Gates that check structure (sections exist, fields are filled) are worth having and worth being honest about — they cannot tell you whether the content is right. Evals sit above them.
• Block the merge. No gate may depend on a human noticing. If the score doesn’t stop the ship, it’s a report, and reports go unread.
• Read back everything. Success responses lie, empty results are facts about instruments, and “zero failures” needs “N cases ran” beside it before it means anything.
• Separate the builder from the checker. The agent (or person) that implemented a change shouldn’t own the tests that judge it — my fleet runs heterogeneous verification for exactly this reason.
• Pilot before platform. Two skills, one afternoon, a merge gate. The platform question answers itself once the pilot has data — and park it with a written unpark trigger, not a vague “later”.

The through-line, and the one sentence to keep: a green tick is only information if the check could have gone red. Evals are how you make an agent stack falsifiable — and falsifiable is what you’re actually selling when you tell a client the thing works.

---

P.S. This guide has a diagram, and the site could not render Mermaid, so publishing it meant teaching the site how. While wiring up the alt text I broke the script that finds the diagrams in the first place — a regex that no longer matched its own fence. Nothing failed. The build stayed green, the page kept serving the previous image, and the only reason I caught it is that I had asked the generator to report how many diagrams it found, and the number was zero.

A check that could not fail, in the tooling for an article about checks that cannot fail. I would say you could not make it up, except I clearly did.