Goodfire discovers reward-hacking signals inside models that can be detected at scale and in real time using activation probes
Models know when they’re reward hacking — and we can catch them at scale
AI BriefingGoodfire Research found activation signals inside models associated with reward hacking and detected them using a simple difference-of-means probe, with performance close to or in some cases better than monitoring based on LLM chains of thought.
Source: Goodfire Research · goodfire.com
Event follow-up1
View all event coverage- Goodfire discovers reward-hacking signals inside models that can be detected at scale and in real time using activation probesGoodfire Research · September 18