Why a Borda Score Is Harder to Game Than an Average
benchmarksevaluationaistatistics
The question
Most leaderboards average a score across tasks. The S1MB decision-model board does something different: it ranks each of 137 benchmarks separately and aggregates the ranks. That is a Borda count.
The board currently carries 197 models, so the choice of aggregation decides who sits where. It is worth understanding before reading any position on it.
Averaging rewards spikes
Suppose a board has 100 tasks and you average the raw scores. One task has scores spread between 0.1 and 0.9, and the other 99 are packed between 0.80 and 0.84.
Improving on that one wide task by 0.3 moves your average by 0.003. Improving on every one of the other 99 by the full width of their spread moves it by about 0.04. Sounds like the many tasks win. But reverse it: a model that is mediocre everywhere and extraordinary on three high-variance tasks can land near the top of the average, because variance, not importance, decided the weighting.
Nobody chose that weighting. It fell out of how the metrics happened to be scaled.
Rank aggregation flattens the scale
Under Borda, each task contributes the same thing: your position among the entrants. Winning a task by 0.3 and winning it by 0.001 both earn the same rank. Being last by a hair costs the same as being last by a mile.
Two consequences follow.
You cannot buy a high placement with a few spikes. To finish first you have to place well on most of the 137 boards at once.
Small real gains count. A method that reliably adds a little on many tasks gains more than one that adds a lot on a few. For an engineer picking a model for a production decision path, that is usually the property worth having.
What rank aggregation gives up
It throws away margin. A board where the top five are separated by 0.001 on every task will still produce a confident-looking Borda ordering. The ranking is real, but the underlying differences may be inside noise.
So read both columns. S1MB publishes a task average next to the Borda score, and the two can disagree. On the current English board the top Borda entry is second on task average, and the model with the highest task average sits third on Borda. Neither column is wrong. They answer different questions.
| Question | Column to read |
|---|---|
| Who places well across the most tasks? | Borda |
| Who has the highest raw scores on average? | Task average |
| Is the gap meaningful? | Neither, you need the per-task spread |
How to read any position on this board
- Check which column the claim is quoting. "First on S1MB" without a column is incomplete.
- Check the date. The board is live and 197 entries move.
- Check coverage. A model scored on fewer tasks is not comparable to one scored on all 137.
- Check whether entries are self-reported. Most public boards mix self-reported and verified numbers, and S1MB notes this.
A note on reading our own entries
Models we publish appear on this board, including the current top Borda entry. We state the column, the date and the model count whenever we cite a position, and we do not describe our own measurement as independent verification. Readers should apply the four checks above to our entries exactly as they would to anyone else's.
Board: S1MB leaderboard, figures read 20 October 2026.