Leading one index and placing fifth on another is the normal case
There is no single axis on which models are ordered. Composite indices manufacture one, and the ordering they produce depends on the weights.

Published
June 28, 2026
Reading time
3 minutes
Perspective
Analysis
Topics
benchmarks · qwen · evaluation
Qwen reported that Qwen3.8-Max ranks fifth on a general intelligence index and first on an agentic index.
That combination reads as tension. It is not — it is what you should expect, and understanding why is more useful than the rankings themselves.
Composite indices are weighted opinions
A "general intelligence index" is an average over chosen benchmarks with chosen weights. Change the weights and the ordering changes. There is no neutral weighting, because there is no fact about how much maths reasoning should count relative to instruction following.
An agentic index averages a different set: tool use, multi-step planning, recovery from errors, long-context tracking. Different skills, different weights, different ordering.
A model tuned hard on tool-use trajectories will do well on the second and unremarkably on the first. That is a coherent description of a system, not an anomaly.
The skills genuinely come apart
The empirical point underneath is that these capabilities are less correlated than people assume:
- Single-turn reasoning rewards getting it right first time from a clean prompt.
- Agentic performance rewards recovering when step four fails — which requires noticing the failure, which is closer to calibration than to reasoning.
A model can be excellent at the first and mediocre at the second, or the reverse. Training emphasis moves them somewhat independently.
What this means when choosing a model
The practical implication is that composite rank is nearly useless for selection. What you need is the sub-scores on the axis matching your workload:
- Building a coding agent? Agentic and long-context sub-scores matter; trivia does not.
- Building a summariser? The reverse.
- Building a research assistant? Retrieval-grounded accuracy and refusal calibration matter more than either.
Reading the top-line number and stopping is how teams end up with a model that benchmarks well and disappoints in production.
The healthy reading of the announcement
A vendor reporting both numbers — including the one where they place fifth — is more informative than one reporting only the flattering rank. The pair tells you something specific about where the model was optimised.
The reflex to treat any non-first ranking as bad news is what pushes vendors toward reporting only their best index. Divergent rankings are the signal, and the field would be better served by more of them being published, not fewer.
Where composite indices are actually useful
Having criticised them, it is worth being fair about what they do well.
Tracking a field over time. A consistent composite, applied to every release, shows whether the frontier is moving. Individual sub-scores are too noisy for that; the average is not.
Screening. If you are choosing between thirty models, a composite is a reasonable first filter to get to five. It is a bad way to choose among those five.
Detecting regressions. A model that drops on a composite between versions has probably lost something real, even if the composite cannot tell you what.
The failure is using them for the last step of a decision rather than the first. A composite rank is a prior, and a weak one; the sub-scores relevant to your workload should overwhelm it.
The reporting practice worth encouraging
The version of this that helps readers is what Qwen did here: report the general index and the agentic index together, including the one where the model places fifth. That pair is informative in a way neither number is alone.
The alternative — reporting only the index where you lead — is technically true and close to useless, and it is the dominant practice precisely because divergent rankings get read as bad news rather than as information.
Continue reading