Decoupling Moment from Event for Video Temporal Grounding

Video temporal grounding (VTG) aims to localize events in untrimmed videos given natural language prompts. Current VTG methods directly inherit the paradigm from 2D object detection, applying rigid IoU-based one-to-one matching between queries and ground truth spans. However, this paradigm fundamentally conflicts with the temporal fluidity of video events: unlike spatial objects, a sub-event (moment) can be semantically aligned with the prompt despite only covering part of the full event span. This matching scheme incorrectly suppresses such moment predictions as negatives, preventing models from learning rich temporal representations. To resolve this, we propose Moment-Event DETR (ME-DETR), a novel framework that embraces the natural part-whole structure of events through dynamic query specialization. During training, after standard Hungarian matching assigns primary event queries to ground truths, we identify other high-confidence predictions within each matched span and designate them as auxiliary moment queries. Through our synergistic supervision strategy, these moment queries are liberated from unreasonable suppression to explore semantically rich sub-events (moments). Extensive experiments demonstrate that ME-DETR establishes new state-of-the-art results, achieving +2.43% mAP improvement on QVHighlights test set without requiring post-processing NMS. Code: https://github.com/zouyuda220/ME-DETR.