Back to Featured
Dwarkesh Patel:Podcast & BlogDwarkesh Patel2026-09-09 00:10September 9

Dwarkesh Patel research: Pretraining progress comes mainly from data improvements

Pretraining progress is mostly coming from data

AI Briefing

Dwarkesh Patel published an experimental analysis training representative model recipes and data corpora from each year between 2019 and 2025 under compute budgets of up to 1e19 FLOPs. He found that data improvements produced a 12.0x increase in compute efficiency, while model improvements produced a 3.7x increase, making the contribution from data approximately 3.24 times that of models.

Source: Dwarkesh Patel: Podcast & Blog · dwarkesh.com

Event follow-up1

View all event coverage

Finished reading the news? Let’s talk about your business.

Leave your contact details, and I’ll get in touch with you personally.

WECHAT

Add me on WeChat

Press and hold or select the WeChat ID below to copy it.