An Agent You Cannot Measure Is an Agent You Cannot Improve

You ran your agent once, it did the thing, and you shipped it. That proves nothing. You tested one input, on one run, and got lucky enough to feel good. The next user is a different input on a different run, and you have no idea what happens then.

The last rung made your agent safe. Safe is not the same as good, and you do not yet know if it is either. Measuring agents is the discipline that turns that anxiety into a number. Without it you are steering blind: every change is a guess, every regression is a surprise reported by an angry user, and "it looked fine when I tried it" is the only evidence you own. This post is about replacing that feeling with evals, the repeatable tests that tell you whether your agent is actually good and whether your last change made it better or worse.

The Same Input Gives Different Answers

Classic software is deterministic. Same input, same output, forever. You write a test that asserts add(2, 2) == 4 and it is true today and true in a decade.

An agent breaks that contract twice over.

First, it is non-deterministic. Ask the model the same question twice and you can get two different answers, because generation samples from a probability distribution rather than reading one off. If you missed why, the foundations post covers the sampling loop underneath. A test that asserts exact-match output will pass on Tuesday and fail on Wednesday with nothing changed.

Second, there is rarely one correct answer. "Summarise this ticket" has a thousand acceptable outputs and a thousand bad ones, and no == will ever separate them. Correctness is a spectrum, not a boolean.

So you cannot assert equality. You have to score quality, across many runs, and reason about a distribution instead of a single result. And every score sits inside a latency-vs-quality trade-off: a slower model or a longer chain of reasoning often scores higher, but users feel every extra second. Measuring only quality while ignoring latency ships something correct that nobody waits for.

A Harness Is the Loop That Makes Measurement Repeatable

You cannot measure by hand. Running ten prompts through a chat window and eyeballing the replies is not a test, it is a mood.

An eval harness is the fixture that automates it: a fixed set of test cases, a runner that feeds each one to your agent, a scorer that grades the output, and an aggregator that rolls the grades into metrics you can compare over time. It is the equivalent of a test suite for probabilistic software.

The reason you need one is change. The moment you tweak a prompt, swap a model, or add a tool, you need to know if you helped or hurt. A harness answers that in minutes: run the suite before, run it after, compare the numbers. No harness means every change is faith.

Build the harness before you optimise anything. Optimising without measurement is just rearranging code until the demo feels better, which is how you ship a regression you will never notice.

Benchmarks Tell You Less Than the Leaderboard Suggests

Benchmarks are standardised evals everyone runs so models can be compared on the same yardstick. You should know the big ones by name, and you should know exactly where each one lies to you.

The trap underneath all of them is win rate, the fraction of head-to-head comparisons a model wins against another. It is a relative score. A model with a great win rate is better than its opponent, which tells you nothing about whether either one is good enough for your task.

Two rules keep benchmarks useful. A benchmark measures the benchmark, not your product: high MMLU does not mean the model handles your customers' refund requests. And contamination is everywhere, because public test sets leak into training data, so a rising score can mean the model memorised the answer key rather than got smarter. Benchmarks pick a model to start from. They never tell you your agent works.

Ground Truth, Hallucination, and Using a Model as the Judge

To grade an output you need something to grade it against. That reference is ground truth: the known-correct answer, or at least a known-correct set of facts, for each test case. Building it is the expensive, unglamorous work of evaluation, and it is the part people skip, which is why their metrics measure nothing.

Ground truth is what lets you catch a hallucination: the model stating something false with complete confidence, an invented citation, a function that does not exist, a fact it made up. You detect it by checking claims against a trusted reference, not by asking the model if it was sure. It is always sure.

But ground truth runs out fast. For open-ended work ("was this summary faithful?", "was this reply helpful?") there is no single string to match. The modern answer is LLM-as-judge: you use a second, often stronger, model to score the first one's output against a rubric. It scales, it is cheap, and it correlates with human judgement well enough to be useful.

It is also biased, and you must design around the biases or your judge lies to you with authority:

• Position bias: in an A-versus-B comparison, judges favour whichever answer they see first. Fix it by running both orderings and averaging.
• Length bias: judges reward longer, more elaborate answers even when shorter is better.
• Self-preference: a model tends to rate its own family's outputs higher than a rival's.

A judge is a measurement instrument, so you have to calibrate it: check its scores against human labels on a sample before you trust it across thousands. An uncalibrated judge feels like measurement and is actually just a second opinion from the same kind of thing you are trying to measure.

One distinction matters once you grade agents and not just answers. Scoring a single reply is prompt evaluation: did this prompt, on this input, produce a good output? Agent-performance evaluation is harder, because an agent is a trajectory, not a string. A run can reach the right final answer while taking six wrong turns, calling a tool it should never have touched, or burning ten times the tokens it needed. So you grade the path as well as the destination: which tools it chose, how many steps it took, whether it recovered from its own mistakes. The final answer is one signal. How it got there is the rest.

The Feeling Is Not the Number

Every level of this ladder before now made your agent do more: reason, use tools, stay safe. This level is the one that tells you whether any of it worked. It is also the least fun, which is why most people skip it and ship on vibes.

Do not. An agent you cannot measure is an agent you cannot improve, because improvement is just a number going up and you have no number. Build the harness, write the ground truth, calibrate the judge, and watch the metric. "It looked fine when I tried it" was never evidence. It was hope wearing the costume of a test. Once the measurement exists, you are ready to put the thing in front of real users and watch it in the wild, which is where shipping begins.

---

Part of the Applied AI Engineering ladder. Previous: Agents You Can Trust. Next: Shipping Agents.