Learning to Mask: Cross-Modal Noise Modulation for Hallucination Mitigation in Multi-modal Large Language Models

Multi-modal Large Language Models have demonstrated im-pressive capabilities in vision-language tasks, yet they remain highly sus-ceptible to hallucinations, in which generated text diverges from the ac-tual visual evidence. Current mitigation strategies, such as self-refinementand specialized decoding, often incur significant latency due to multi-round iterations or reliance on high-performance auxiliary models, limit-ing their real-time practical utility. In this paper, we systematically ana-lyze the hidden states of MLLMs across the entire inference pipeline andreveal that hallucination-inducing redundant features persist not onlyin image tokens during the prefill stage but also propagate into Imageand Answer Value Caches during decoding. Motivated by the observationthat Gaussian noise injection can effectively suppress these redundancies,we propose HalMask (Hallucination Feature Masking), a lightweightand real-time self-correction framework. HalMask employs specializedImage and Text Noise Modulation Modules, optimized via a dynamic reg-ularization term, to accurately discern and mask hallucination-triggeringfeatures. During inference, it applies a dynamic Value Cache maskingmechanism that continuously filters redundant information across bothgeneration phases without disrupting critical semantic cues. Extensiveexperiments on multiple benchmarks demonstrate that HalMask signifi-cantly reduces hallucinations while introducing only marginal computa-tional overhead.