Epoch AI releases the InnovationEval benchmark: frontier models achieve only 15% of the SDPO gains from human papers
Epoch AI launched the InnovationEval benchmark, which tests whether AI can independently reproduce machine learning innovations from human research papers, using Self-Distillation Policy Optimization (SDPO) as the reference method.
Why it's recommended:Using a benchmark developed in-house, the original article quantifies the gap between frontier models and end-to-end AI research and development capabilities. It also documents exaggerated results and reward hacking, providing useful context for assessing progress in AI-driven research automation.