A discussion in r/MachineLearning on the current state of language models and preference-based rankings is a good prompt to be precise about what these leaderboards measure.

They are not fake. They capture something real and hard to get any other way. But the gap between "raters preferred this" and "this is better" is wider than the presentation implies, and the gap has structure.

What a preference vote actually records

A rater sees two responses and picks one. That choice reflects, in rough order of influence:

1. Formatting and scannability. Structure, length, whether the answer is visible without reading.
2. Tone match to what the rater expected.
3. Apparent confidence. Hedged answers lose to assured ones, independent of correctness.
4. Actual correctness — but only when the rater is positioned to judge it.

Point 4 is the one everyone assumes is dominant. For a large fraction of prompts, the rater cannot evaluate correctness at all, because they do not know the answer either.

Where the drift is predictable

Preference ranking systematically rewards properties that correlate with quality unevenly:

• Length, up to a point, then against.
• Confident phrasing, which is precisely wrong for questions with uncertain answers.
• Immediate legibility over completeness.

None of these is a scandal. They are properties of asking humans to judge quickly. But they mean a model can climb a preference leaderboard by getting better at presentation while staying flat on substance — and the leaderboard cannot distinguish that from real improvement.

What they are genuinely good for

Preference rankings are the best available instrument for a specific question: which model do people prefer interacting with, across a broad and messy prompt distribution.

That question matters. It is a product question, and it is close to unanswerable by static benchmarks, which is why these leaderboards took over.

The error is treating an answer to that question as an answer to "which model is more capable," which is a different question with different methods.

How to read them without being misled

• Look at margin, not rank. A 15-point gap on an Elo-style scale is noise dressed as an ordering.
• Look at category breakdowns where available. Aggregate scores hide that a model may lead on writing and trail badly on maths.
• Cross-check against verifiable benchmarks for anything where a right answer exists.
• Treat any leaderboard where the model developer controls the prompt distribution as marketing, not measurement.

The healthiest version of the current situation is one where preference rankings and verifiable benchmarks are both reported and neither is asked to do the other's job.