Back to Featured
Apple Machine Learning Research2026-07-02 08:00July 2

Study of robustness and chain-of-thought consistency in RL fine-tuning of VLMs

On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs

AI Briefing

Reinforcement learning (RL) fine-tuning is extended to vision-language models (VLMs). The study finds that simple text perturbations—misleading captions or incorrect chains of thought (CoT)—significantly reduce model robustness and confidence, with open-source models showing a greater decline. Closed-source models exhibit similar failure patterns, but have stronger robustness and reasoning consistency. Further analysis reveals a trade-off between accuracy and faithfulness: fine-tuning improves benchmark accuracy while eroding CoT reliability and robustness to contextual changes; adversarial training can improve robustness but cannot prevent faithfulness drift. Introducing faithfulness-aware rewards restores alignment between answers and reasoning, but training tends to collapse into shortcut strategies when combined with robustness enhancements. These findings highlight the need to jointly consider correctness, robustness, and faithfulness in visual reasoning.

Source: Apple Machine Learning Research · machinelearning.apple.com

Finished reading the news? Let’s talk about your business.

Leave your contact details, and I’ll get in touch with you personally.

WECHAT

Add me on WeChat

Press and hold or select the WeChat ID below to copy it.