NarrativeTrack: Evaluating Entity-Centric Reasoning for Narrative Understanding

Multimodal large language models (MLLMs) have achieved impressiveprogress in vision-language reasoning, yet their ability to understand temporallyunfolding narratives in videos remains largely underexplored. Narrative under-standing requires more than recognizing isolated events: models must maintaincoherent representations of who is doing what, when, and where across scene tran-sitions and temporal gaps. We introduce NARRATIVE T RACK, the first benchmarkto evaluate narrative understanding in MLLMs through fine-grained entity-centricreasoning. Unlike existing benchmarks limited to short clips or coarse scene-levelsemantics, we decompose videos into constituent entities and evaluate modelsusing a Compositional Reasoning Progression (CRP), a structured frameworkthat progressively increases narrative complexity across three dimensions: en-tity existence, entity changes, and entity ambiguity. This progression requiresmodels to move beyond local perception to reasoning about entities’ temporalpersistence, state changes, and fine-grained perceptual disambiguation. To enablescalable benchmark construction, we develop a fully automated entity-centricpipeline that extracts temporally grounded entity representations and providesthe foundation for CRP. Evaluations of state-of-the-art MLLMs reveal that ex-isting models struggle to maintain coherent entity representations under visualtransitions and temporal dynamics. Open-source general-purpose MLLMs exhibitstrong perceptual grounding but weak temporal continuity, while video-specializedMLLMs capture temporal context yet frequently hallucinate entities’ contexts.These findings uncover a fundamental trade-off between perceptual grounding andtemporal reasoning, indicating that narrative understanding emerges only fromtheir integration. NARRATIVE T RACK provides the first systematic framework todiagnose and advance temporally grounded narrative comprehension in MLLMs.