HERO: Enhancing Multimodal Faithfulness via Dynamic Entropy-Aware Reinforcement Learning

Chain-of-Thought fine-tuning enhances the reasoning capabilities of Large Vision-Language Models, but can paradoxically exacerbate hallucinations, revealing a reasoning-faithfulness trade-off. We identify a key cause of this phenomenon as the “Confidence Trap”: under visual ambiguity or degradation, models may produce low-entropy, high-confidence errors instead of expressing appropriate uncertainty. This failure mode challenges conventional mitigation strategies that rely on uncertainty cues or contrastive decoding signals, since confident hallucinations can remain highly stable under visual perturbations. To address this problem, we propose Hallucination-Entropy Regulated Optimization (HERO), an on-policy reinforcement learning framework for mitigating confident hallucinations. HERO introduces a dynamic entropy-aware objective that assigns larger optimization weights to low-entropy erroneous outputs, and further improves training efficiency through variance-gated sample selection that focuses updates on informative hard negatives. Experiments on POPE, THRONE, and AMBER show that HERO achieves strong overall hallucination mitigation, substantially improves truthfulness in critical low-entropy regions, and maintains competitive general reasoning capabilities. These results suggest that explicitly targeting confidence-misaligned errors is an effective direction for improving multimodal faithfulness in CoT-style LVLMs.