Nathan Lambert discusses Artificial Analysis v5 and the predicament facing benchmark creators
Artificial Analysis v5 and the benchmarker's dilemma
AI BriefingNathan Lambert analyzes the Artificial Analysis Intelligence Index v5 update and the evaluation challenges it brings. AAv5 divides the index equally among four categories—knowledge work, coding, science, and general—at 25% each. The leaderboard shows that 8 closed-source models outperform Kimi K3.1 Pro, the strongest unreleased open-weight model.
Source: Nathan Lambert: Interconnects · interconnects.ai
Event follow-up2
View all event coverage- Tencent Hunyuan thanks Artificial Analysis for its Hy4 evaluation; model scores 35 on Intelligence Index v5Tencent Hunyuan · 11 days ago
- Google DeepMind releases Gemini 4 Carbon, a frontier modelGoogle DeepMind:Blog · 19 days ago