ChronusOmni: Improving Time Awareness of Omni-Modal Large Language Models

Time awareness is a fundamental ability of omni-modal largelanguage models, especially for understanding long videos and answeringcomplex questions. Previous approaches mainly target vision-languagescenarios and focus on the explicit temporal grounding questions, suchas identifying when a visual event occurs or determining what event hap-pens at a specific time. However, they often make insufficient use of theaudio modality, and overlook implicit temporal grounding across modal-ities (for example, identifying what is visually present when a characterspeaks, or determining what is said when a visual event occurs), de-spite such cross-modal temporal relations being prevalent in real-worldscenarios. In this paper, we first formally define the audiovisual tem-poral grounding task to systematically encompass both explicit andimplicit cross-modal temporal relations. To tackle this task, we proposeChronusOmni, an omni-modal large language model designed to en-hance temporal awareness for both explicit and implicit audiovisual tem-poral grounding. Specifically, we interleave text-based timestamp tokenswith visual and audio representations at each time unit, enabling uni-fied temporal modeling across modalities. Furthermore, to enforce correcttemporal ordering and strengthen temporal reasoning, we incorporate re-inforcement learning with specially designed reward functions. Moreover,we construct ChronusAV, a temporally-accurate, modality-complete, andcross-modal-aligned dataset to support the training and evaluation of theaudiovisual temporal grounding task. Experimental results demonstratethat ChronusOmni achieves state-of-the-art performance on audiovisualtemporal grounding datasets such as ChronusAV and LongVALE, out-performing the second-best results by 17.3% and 28.8%, respectively, anddelivers top-tier performance on visual-only temporal grounding datasetsincluding Charades-STA and ActivityNet. These achievements highlightthe strong temporal awareness of our model across modalities, while pre-serving its robust capabilities in general video and audio understanding.