What a thumbs up can't tell you about your AI product

Almost every AI product ships with the same two icons under its outputs: thumbs up, thumbs down. At a lot of companies, those two icons are the only human judgment the team ever collects.

That is an eval, of a narrow kind. It measures how a user reacted to the one output you showed them. It can't tell you whether that output did what they came to do, and it arrives as one bit of signal from a small, self-selected slice of your users, with little attached to act on.

What thumbs data actually gives you

In practice, thumbs data has four problems.

It rates the reaction, not the outcome. Users judge an answer by how it reads in the moment. A fluent, confident answer that invents a refund policy can earn a thumbs up, because the user has no way to know the policy doesn't exist. They find out later, when support says no, and the rating never changes. They also only see the answer you gave them, so they can't tell you a better one was possible.

It's sparse. Most users never click the icons. Most outputs get no rating at all, so the bulk of your product's quality is invisible to you.

It's skewed. The people who click tend to be the furious and the delighted. Everyone in between, which is most of your users and many of your quality problems, stays silent.

It's ambiguous. A thumbs down can mean the answer was wrong, or right but rude, or right but slow, or too long, or badly formatted, or that the user misclicked. You get the same single bit for a made-up price and a paragraph that ran long. One of those is a five-alarm fire. The other is a style tweak. Your data can't tell them apart.

Say the dashboard shows 4% of rated outputs got a thumbs down this week, up from 3%. Nobody can say whether that's signal or noise, which model change caused it, which dimension of quality slipped, or what to fix. The metric moved, and the team argues about vibes.

Google never relied on clicks alone

If passive user feedback were enough to measure quality, Google would be the one company that never needed anything else. Every click, every rephrased query, every back-button press on billions of searches a day is a tiny thumbs signal. Few companies have ever held more behavioral feedback.

Google has all of it and still relies on a standing network of more than 10,000 trained Search Quality Raters around the world, working from a published rubric that runs 182 pages.

Those raters score what they see instead of just browsing and reacting. The guidelines define quality as separate, named dimensions: a Needs Met scale that runs from "Fully Meets" down to "Fails to Meet," and Page Quality criteria built on experience, expertise, authoritativeness, and trust. The Needs Met scale scores each result against what the searcher is trying to get done, which is the question a thumbs click can't reliably answer. Raters also do side-by-side comparisons. They see two versions of a results page, one with a proposed change and one without, and say which they prefer and why.

In 2023, Google ran 719,326 search quality tests and 124,942 side-by-side experiments, and made 4,781 launches. Search quality tests outnumbered launches about 150 to 1.

Rater scores don't directly change rankings. Google uses them as a benchmark. Human judgment is the measuring instrument, and proposed algorithm changes get held up against it before launch.

The company sitting on all that behavioral data ran 16,871 live traffic experiments in 2023 and still kept its human raters on top of them.

I've spent a good part of my career on A/B testing, first helping build infrastructure for running experiments on Google Search, then launching Feature Experimentation at Optimizely. Live experiments are very good at measuring behavior. Behavior and quality are related, but they're not the same. Clicks tell you what happened. Raters tell you whether it was good.

Frontier labs figured this out, too

The same lesson is playing out at the model layer, with bigger checks.

The labs building frontier models have more automated benchmarks than anyone. They're also spending heavily on people. Mercor recruits doctors, lawyers, bankers, and engineers to evaluate model outputs for the top AI labs. As of May 2026, the company says it pays out over $2 million a day to more than 30,000 weekly active contractors. In September 2025, its CEO, Brendan Foody, put the median pay rate in its marketplace at $95 an hour. He argues that the labs' "primary bottleneck" to improving models is measuring what success looks like.

A generic benchmark can't tell a lab whether a model's oncology answer is subtly wrong. An oncologist scoring it against criteria can. The labs have more automated evaluation than any product team will ever have. They're still paying expert rates for structured judgment from people qualified to give it. That spending tells you where the scarce input is.

If the lab that built your model is buying human judgment from experts paid a median of $95 an hour, why is your product collecting it with two free icons?

Mostly because the tooling in between was built for someone else. Most data-labeling tools were built for annotation pipelines: you bring your own labelers or rent a generic workforce. Developer eval platforms live in the trace: useful for logging outputs and running LLM judges, but the human in that loop is usually you, reviewing your own transcripts. Expert networks price judgment for frontier-lab budgets. Between two icons in your UI and a $95-an-hour expert workforce, very little has been built to turn judgment from people like your own users into a rubric and a judge you can rerun.

So your AI product has the same measurement problem as search results and frontier models: subjective, multidimensional quality that shifts with every change, and behavioral signals that can't explain themselves. Google answers it with trained raters, and the labs answer it with expert evaluators. Many product teams are still answering it with two icons and a dashboard.

What structured judgment looks like

Translate Google's method to your product and you get three things a thumbs icon rarely collects: ratings against a rubric, preferences between outputs, and rationales.

Rubrics break quality into the dimensions it actually has. Accuracy, helpfulness, tone, and safety are different properties, owned by different parts of your system, fixed in different ways. When evaluators score them separately, "quality dropped" becomes "accuracy held, tone regressed after the new system prompt." That's a finding you can assign to someone. The rater guidelines are Google's rubric for search. Your product needs its own, specific enough to measure against.

Preferences address a known weakness of human judgment: people are inconsistent at absolute scores and steadier at comparisons. Ask someone to rate an answer 1–5 without anchored definitions and you get noise, because the distinction between a 3 and a 4 is subjective and inconsistent. Show them two answers side by side and ask which is better, and they have something concrete to anchor on. Google ran 124,942 side-by-side experiments in 2023, on top of its absolute quality ratings. The question you actually have is whether the new prompt beat the old one. A thumbs rating has no counterfactual. A preference has one built in.

Rationales are the part teams skip and the part worth the most. A downvote says "bad." A rationale says "the second answer invented a cancellation policy that doesn't exist on your pricing page." One is a data point. The other is a bug report. Google asks its raters which results they prefer and why. With the why attached, every low score comes with something to fix.

Collecting all of this also pays off later, because it's what lets you automate evaluation. Sooner or later you'll want an LLM judge scoring outputs continuously. But a judge can only be as consistent as the human judgment it's calibrated against, and thumbs are too sparse and ambiguous to calibrate against. You calibrate against a golden set: outputs scored by real people, on your rubric, with their reasoning attached. Google's raters play the same role. They benchmark the algorithm instead of steering it directly. Skip the human data and your automated judge will score thousands of outputs against the wrong standard. Your eval numbers can improve while your product doesn't.

Do this well and the team stops debating vibes and starts arguing about specific failures, with scores attached.

Evals and user research do different jobs

Evals and user research get confused because the tools look similar and people use the words interchangeably.

Research methods (AI-moderated interviews, surveys, usability tasks, video observation) answer questions about your users' world. What do they need? Why did they churn? How do they actually work? The questioning is often adaptive and open-ended, probing the way a qualitative researcher does, because the goal is discovery. You don't know what you're looking for yet. The output is understanding.

Human evals answer a question about your product's world: is this output good by the criteria you defined, and is it better than last week's? The structure is fixed on purpose. Same rubric, same task, comparable results across time and across model versions. The output is a measurement. Google's raters work as instruments, measuring the search experience against a standard rather than researching searchers.

Mix them up and you get failure in both directions. Run your eval like an interview and you get vivid anecdotes you can't compare release to release. Run your research like an eval and you get tidy numbers with no discovery in them, scores against a rubric nobody validated.

The two feed each other, in order. Research surfaces the requirements: what your users need, and what quality means to them. A thumbs icon can't surface either, because it only records a reaction to what you already built. The requirements become the rubric. Evals then measure the real experience against that rubric, every release, and what they find becomes the next round of research questions.

Run Google's method with your own users

The lesson for a product team is Google's method, not the size of its rater network. Google treats quality measurement as a staffed, deliberate function. It recruits evaluators, trains them on a rubric, and has them compare versions side by side before a change launches. It never left that job to a widget on the results page and whoever felt like clicking.

Google's 10,000-plus raters are that method at the scale of Search. A product team doesn't need that many. It needs the same structure, with one difference. Google's raters stand in for searchers. A product team can go a step closer and have the people it's building for do the judging. That model used to mean building and staffing your own rater program. Now a team can recruit hundreds of its own target users before launch, record them using the product, turn what they say and do into a rubric, and build a golden set from their scores, preferences, and reasons.

When your product was deterministic, you could test it before shipping and trust it after. AI products don't work that way. Every model update, prompt change, and retrieval tweak shifts output quality in ways no unit test catches. Before a change reaches live traffic, human judgment collected with enough structure to act on is the best instrument you have for what your users will experience.

Google has more click data than almost anyone. It measures quality with people anyway. A thumbs icon is a suggestion box bolted to a system that changes underneath you every week. Keep the icons. Just don't let them be the only source of real user feedback in your evals.