What are AI evals? Five types, and the one most teams are missing

AI evals are tests that score what an AI product produces against a definition of good. There are five types, and a team can run every one and still lose users.

Your company shipped an AI feature, maybe an in-app agent. The AI team did the responsible thing: unit tests, deterministic evals, an LLM judge in Braintrust or something homegrown, end-to-end tests that drive the app like a script. The dashboards are green. Retention is soft anyway, and nobody in the room can say which user scenarios are failing. So the evals are broken?

No. They're doing exactly what they were built to do. Each type of eval answers one question, and "are user needs being met" isn't on the list. Walk through the types and the gap is easy to see. So is the fact that you fill it alongside the evals, not instead of them.

What is an AI eval?

Every eval has the same three parts: the thing being graded (an answer, a screen, a whole session), the definition of good (a reference answer, a rule, or a rubric), and the grader (code, a model, or a person). Every type below is a different choice for those three parts. The part that matters most is the definition, which is why Anthropic's guide to agent evals lands the responsibility where it belongs: the people closest to product requirements and users are best positioned to define success. That's you, not the model.

Two things separate evals from the testing you already know. Unit tests ask whether the code broke; evals ask how good the output is. And because a model gives different answers to the same input, an eval reports a rate, not a pass. Teresa Torres's guide for product teams puts it plainly: LLMs are probabilistic and code is deterministic, so the old tests measure the wrong thing.

1. Unit tests and deterministic evals

The question they answer: did it break, and did it match?

Rules run by ordinary code, plus a fixed set of inputs with known-correct outputs. The response is valid JSON. The answer came back under two seconds. The quote the assistant attributed to the source actually appears in the source. The router sent the billing question to billing. Torres's example is a hallucination guard that string-matches every quoted passage against the document it supposedly came from.

Cheap, instant, objective, and blind to quality. A response can pass every assertion and still be useless to the person who asked. And the moment more than one answer is valid, which is the moment for any search page, summary, or conversation, the reference answer is the builder's opinion frozen in a file. These are the floor, and a floor is not a bar.

2. End-to-end tests

The question they answer: does the whole flow still run?

Something drives the product the way a user would, start to finish, and checks that it got somewhere. The Bolt AI team, which started building evals after their agent already had a wide audience, built a system in three months that runs the agent, grades output with static analysis, uses browser agents to test the apps it builds, and adds LLM judges for behaviors like instruction following. The browser agent is the part to notice: it's the closest thing on this list to a user.

Closest, not the same. A browser agent follows a scripted path and reports whether the path broke. It's bug catching at the level of the whole product, which is valuable and which nobody should stop doing. It isn't a set of scenarios your product team wrote down in the language of the people who use the thing, and it can't tell you whether the path it walked was the one that mattered to them.

3. LLM-judge evals

The question they answer: does it meet the bar the team wrote down?

A second model reads the output and scores it against a rubric. This is how teams grade the open-ended work at scale, and it's where the well-resourced teams have gotten genuinely good. Descript's agent edits video, so they built evals around three dimensions of a good editing session: don't break things, do what I asked, and do it well. They graded by hand first, then moved to LLM graders with criteria defined by the product team and periodic human calibration, and now run two separate suites, one for quality benchmarking and one for regression. Anthropic's own split is the same: capability evals that ask what the agent can do, regression evals that ask whether it still does everything it used to.

Emulate that. Then notice what the rubric grades against. "Don't break things" and "do what I asked" are system questions with system answers. "Do it well" is where the user's judgment enters, and in every stack like this one it's judged by the team's criteria, calibrated to the team's reading of a sample of transcripts. Every guide says calibrate the judge to humans. Hamel Husain says it, Anthropic says it, Torres says it. Then they stop. Which humans? Recruited how? Against what? A judge is a rubric with a throughput budget. Change the rubric and you change what good means.

4. Human review

The question it answers: does a person think it's good?

People read the outputs and rate them: the team grading transcripts by hand, an expert spot check, a side-by-side comparison. Anthropic's advice on this is three words and an exclamation mark, read the transcripts, and Descript's manual grading is where their whole system started. This is the ground truth every judge gets calibrated against, and it's how Google has graded Search for two decades, with a rater program that runs on a 182-page rubric and thousands of side-by-side judgments a year, on top of every automated metric it tracks.

Product teams have their own version of this, and it predates AI: user research. Interviews, usability sessions, watching five people try the feature. It's the deepest signal on the list and the slowest, which is why it happens once a quarter and never for a prompt change. In both versions, "human" is doing a lot of work. The humans reading transcripts are the team. The humans in a usability study are users, but five of them, once. A domain expert can tell you the answer is correct. Only the person who came with the need can tell you it was the answer they needed, and you need to hear that from more than five of them, more than once.

5. Production signals

The question they answer: what are users doing with it?

Thumbs up and down, regenerations, edits to the output, follow-up questions, retention. Torres calls them customer feedback evals and notes they're the ultimate validation and the hardest to act on. They're real users, at last. They're also late and ambiguous: a thumbs-down on a summary says nothing about which sentence failed, and a retention dip six weeks after a model swap says even less about why. This is the type that produces the sentence every product leader has said this year. Users are leaving and we don't know why.

What's missing from a stack that has all five

Look at what each type grades against. Unit tests and deterministic evals grade against the spec. End-to-end tests grade against a scripted path. The LLM judge grades against the team's rubric. Human review grades against the team's reading. Four types, and every one measures whether the system did what you meant. Production signals finally involve the user, after the fact and without the reason.

Search is the cleanest illustration of why that's not enough, which is why Google built the hybrid it did. The number of possible results pages for the queries people type is effectively unbounded, and whether a page is good depends on what the person meant, which no metric can see. So humans define good, in detail, and automated systems carry that definition across scale. Our co-founder John Provine helped build those human evaluation systems, the ones used to assess thousands of changes to Search every year. Every LLM-powered feature imports search's problem into your product. The moment the output can be any of a million things, and the right one depends on who asked, "did it do what we asked" stops being the question. "Did this person get what they came for" is.

Anthropic's guidance for conversational agents shows the field reaching for exactly this. It recommends grading end states plus rubrics that capture task completion and interaction quality, and it notes that teams often use a second LLM to simulate the user. A simulated user is a stand-in for the one participant the stack doesn't have.

The eval that sits alongside the other five

Call it a user scenario eval. A set of scenarios your product team wrote down, in the language users would use ("I searched 'vintage Levi's jacket size M' because I want one that actually fits my style"), graded on whether the need was met, by a rubric anchored in what real users said when they tried those scenarios, run by a judge calibrated to those users. It runs offline, on the same scenario set every time, before users see anything, so a score on Tuesday is comparable to a score on Friday. Its output is a view of your product in the language of users and product people: which scenarios are working, which are failing, and where you stand on user needs being met.

It doesn't replace anything above. The unit tests keep catching breakage. The judge keeps grading instruction following. The browser agent keeps walking the path. This one answers the question the others were never built for, and it does it before the retention dip, on the scenarios you can name, with reasons attached. That's the difference between "users are leaving" and "the refund scenario fails for anyone with two orders, and here's what they said."

You need all three, at some scale

Product teams hear "evals" and assume it's the AI team's job, then hear "user research" and assume it can't keep up. Both are half right. The honest picture has three layers, and none of them substitutes for the others.

User research is deep, slow, and occasional. Keep it for the questions that need a conversation. Engineering evals are fast, cheap, and system-shaped. Keep all of them, on every commit. User scenario evals sit between: user judgment, at eval cadence. Real people set the bar once, a calibrated judge holds it on every release, and you re-check the bar as users change. If you're a product manager deciding what to fund, that's the answer. All three, at whatever scale you can afford, and the middle one is the layer most teams don't have.

Humans inform, judges scale, production recalibrates

Real users define good. Recruit people who match your customers, put them through the scenarios that matter, and record what they do and say. Their ratings and reasons become the rubric: not how good the AI sounds, but how well the need was met. Here's what one looks like.

An autograder calibrated to them scales it. An LLM judge is tuned until it agrees with those users, then run across hundreds or thousands of scenarios, offline, before anything ships. A launch review, a regression check, or a competitive benchmark becomes a score you can re-run on every change.

Production recalibrates. Live users surface new scenarios and new failures. Those flow back into the rubric, the judge gets re-checked against fresh human judgment, and the loop goes around again. Humans once, judges always, with the once on a schedule.

This is what we built Grade to do. We recruit your users and run the scenarios, we help you turn what they say into a rubric, and we deliver an autograder calibrated to it, one you can run anywhere a decision gets made and next to every eval your AI team already has. Ad hoc user research can't keep up with AI development; you can't run a new study for every product update. A calibrated grader can, because the humans went in once, up front.

Which type decides

All five stay. The question is which one gets to say a release is better. The floor types say it didn't break. The judge says it followed instructions. Production says people came back, or didn't. Whether this version is better for the people it's for belongs to a grader that agrees with those people. If your stack today is the AI team's four plus a dashboard, you have "is it working" covered four ways and "is it good" covered zero.

How good is your product? For the argument behind that question, read How good is your AI product? And if you want a grader calibrated to your own users sitting next to the evals you already run, join the Grade waitlist.