Benchmark5d ago
Artificial Analysis Releases HealthBench & MLCR-AA: Exposing Severe Verbosity Bias in Medical AI Rankings
Independent AI evaluation platform Artificial Analysis officially released HealthBench Professional alongside the length-normalized MLCR-AA index (@ArtificialAnlys). The benchmark revealed a pervasive industry blindspot: LLM-as-a-Judge evaluations suffer from acute 'verbosity bias', disproportionately rewarding longer generations regardless of medical conciseness. In raw HealthBench scoring where Anthropic outputs were 4–6x longer, Claude Opus 5.5 led at 71.4%. However, once length bias was scientifically normalized via MLCR-AA, GPT-6 Astra (max) tied Claude Sonnet 5.5 for #1, while Claude Opus 5.5 plummeted from #1 down to #6, setting a new rigorous standard for medical model evaluation.
Benchmark52d ago
LMArena: Claude Fable 5.1 (Max) #1 on Agent Arena (+15.8% net) at $4.14 median cost/task
Arena.ai says Anthropic Claude Fable 5.1 (Max) is #1 on Agent Arena across 6.7k+ real-world long-horizon agentic sessions with +15.8% net improvement, redrawing the price-performance frontier at a $4.14 median cost/task. Implicit praise-vs-complaint leads by +42.5%, confirmed success +22.4%, bash recovery +13.1%, and no tool hallucinations; GPT-6 Astra traces are still coming in.
Benchmark41d ago
Vals Live Terminal-Bench 4.0: GPT-6 Astra Leads at 57.1%
Vals AI launched Terminal-Bench 4.0: 66 new end-to-end terminal tasks (ship a service, prove a theorem, train a GPU kernel, write a forensic report, and more). Median expert effort is about 4 hours; grading is strict (full verifier suite or zero), and scores are avg@3. GPT-6 Astra won at 57.1%, 7.6pp ahead of Claude Fable 5.1 (49.5%) and 11.6pp ahead of Claude Opus 5 (45.5%). No other model cleared 30%; fourteen of 27 models scored zero on both hardware and media. Version 4.0 spans seven categories—software, science, ML, operations, hardware, security, and media—with roughly three-quarters of tasks outside traditional software work and no overlap with v2.1.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.