Press Space to continue
Current AI benchmarks are unreliable for measuring model progress
Press Space to continue
The state of public AI benchmarking is dire and is undermining our ability to understand how good AI is now. Most famous measures are maxed out, and, as this paper shows, the non-saturated benchmarks are riddled with so many errors that they vastly underestimate AI abilities.
open on x · 1 viewViews and clicks count once per visitor per day. Clicks are opens of this post on X.