Back to Featured
Apple Machine Learning Research2026-08-20 08:00August 20

Hybrid pretraining scaling laws under data constraints

Scaling Laws for Mixture Pretraining Under Data Constraints

AI Briefing

Apple's machine learning research team systematically studied the trade-offs involved in mixed pretraining with scarce target data and general-purpose data through more than 2,000 language model training experiments. The study found that an excessively low proportion of target data leads to insufficient exposure to the target domain, while an excessively high proportion causes diminishing returns and even overfitting due to too many duplicate samples, providing quantitative guidance for mixed pretraining strategies under data constraints.

Source: Apple Machine Learning Research · machinelearning.apple.com

Finished reading the news? Let’s talk about your business.

Leave your contact details, and I’ll get in touch with you personally.

WECHAT

Add me on WeChat

Press and hold or select the WeChat ID below to copy it.