Beyond Where to Look: Trajectory-Guided Reinforcement Learning for Multimodal RLVR

Recent advances in Reinforcement Learning with VerifiableRewards (RLVR) for multimodal large language models (MLLMs) havemainly focused on improving final answer correctness and strengthen-ing visual grounding. However, a critical bottleneck remains: althoughmodels can attend to relevant visual regions, they often fail to effectivelyincorporate visual evidence into subsequent reasoning, leading to reason-ing chains that are weakly grounded in visual facts. To address this issue,we propose Trajectory-Guided Reinforcement Learning (TGRL), whichguides the policy model to integrate visual evidence into fine-grained rea-soning processes using expert reasoning trajectories from stronger mod-els. We further introduce token-level reweighting and trajectory filteringto ensure stable and effective policy optimization. Extensive experimentson multiple multimodal reasoning benchmarks demonstrate that TGRLconsistently improves reasoning performance and effectively bridges thegap between visual perception and logical reasoning.