Learn about this first
Goodfire Research published a paper proposing the mechanistic shaping framework, which uses reliance shaping and expression shaping during training to penalize a model’s reliance on or expression of specific internal concepts. The goal is for models to solve problems correctly rather than merely produce correct answers.
According to Goodfire, the framework has been applied in three scenarios: chest X-ray interpretation, reward hacking in large language models, and sycophancy. The publicly available materials are currently limited to the paper and a blog summary, with no independently verifiable performance data provided yet.
AI-generated from reports · updated 11 days agoLatest developments
Goodfire proposes a mechanistic shaping framework to use internal representations to suppress models from taking shortcuts
Event progress1
Goodfire Research Releases Mechanistic Shaping Framework to Improve Model Alignment and Monitoring
Goodfire Research: Goodfire Proposes a Mechanistic Shaping Framework to Suppress Model Shortcut-Taking Using Internal Representations