DARE to Mitigate Hallucination: Dual-path Auto-Regressive-aware Editing

Large vision-language models (LVLMs) have recently achievedremarkable progress across multimodal tasks, yet object hallucinationremains a persistent challenge where models generate descriptions in-consistent with the visual input. Recent work mitigates hallucinationsthrough training-free representation editing, typically by constructinghallucination-related directions from teacher-forcing (TF) contrasts be-tween hallucinated and truthful responses. However, LVLMs operatethrough autoregressive (AR) decoding during generation, raising thequestion of whether TF-based analysis fully reflects the generation dy-namics that lead to hallucinated outputs. In this paper, we analyzethe relationship between TF-based editing and AR generation behaviorand find that TF-based editing alone may be insu!cient to captureboth decoding dynamics and multimodal interactions associated withhallucinations. To address this limitation, we propose DARE (Dual-path Auto-Regressive-aware Editing), a hybrid hallucination editingframework that integrates two complementary contrast pathways: textualcontrasts and image contrasts, together with autoregressive-aware rep-resentation signals. Specifically, DARE constructs hallucination editingdirections from (1) TF-based textual contrasts, (2) AR-aware represen-tation transitions during decoding, and (3) controlled visual di"erencesbetween paired images. Extensive experiments on multiple LVLM hal-lucination benchmarks demonstrate that DARE consistently reducesobject hallucinations while preserving multimodal perception capabil-ity and inference e!ciency. Our implementation code is available athttps://github.com/KU-VGI/DARE.