Anthropic Has Claude Train Models Autonomously to Mitigate Alignment Failures
Automated researchers can reliably mitigate alignment failures
AI BriefingAnthropic had Claude autonomously train models to mitigate 10 types of alignment failures, including deception and sycophancy. All showed significant reductions in the safety gap from perfect performance without impairing general capabilities, and the method remained effective on a model 4.7 times larger than the optimization target. Claude also outperformed 28 human safety researchers; its best method in deception scenarios was 20% better than the best human approach.
Source: Anthropic: Research · anthropic.com
Event follow-up1
View all event coverage- Anthropic Has Claude Train Models Autonomously to Mitigate Alignment FailuresAnthropic: Research · August 29