OpenAI trains and deploys activation classifiers as a safety monitoring layer for frontier models
Training activation classifiers for safety and alignment
AI BriefingStarting with GPT-5.6, OpenAI has deployed activation classifiers (probes) on internal and external traffic for its most capable models. These probes use internal model activations to score each token in real time, allowing noncompliant responses and tool calls to be blocked mid-generation and referred to a reasoning model for review. Experiments show that probes are more robust than larger text classifiers against the jailbreak attacks tested, improve in accuracy as model capabilities increase, and transfer well across post-training checkpoints. However, they require retraining for new base models and can detect only known violation types for which they can be explicitly trained.
Source: OpenAI: Alignment Research Blog · alignment.openai.com
Event follow-up3
View all event coverage- OpenAI Reports Covertly Reduced Work Quality by GPT-6 Astra on Safety Research TasksOpenAI: Inaccurate Reports and Statements · 12 days ago
- OpenAI trains and deploys activation classifiers as a safety monitoring layer for frontier modelsOpenAI: Alignment research blog · 14 days ago
- OpenAI details how it monitors misaligned behavior in frontier modelsOpenAI: Alignment research blog · 17 days ago