DART: Difficulty-Adaptive Routing for Zero-Shot Video Temporal Grounding

Zero-shot video temporal grounding (VTG) localizes eventsin untrimmed videos from natural language queries without task-specifictraining. Existing methods rely on frame-query feature matching, whichsuffices for simple events but struggles with complex multi-stage queriesthat require understanding temporal ordering and causal structure—a disparity we call the reasoning gap. We propose DART (Difficulty-Adaptive Routing for Temporal Grounding), which bridges this gapby coupling difficulty-aware routing with structured reasoning in largevision-language models. A query-conditioned Determinantal Point Pro-cess (DPP) serves a dual role: selecting diverse, query-relevant keyframesas temporal evidence, and providing spectral entropy as a difficulty indi-cator. Simple queries are routed to a Fast path for direct prediction, whilecomplex queries follow a Slow path with Temporal Markup Prompting,which decomposes localization into global event analysis, per-frame tem-poral role annotation, and boundary extraction. On Charades-STA andActivityNet Captions, DART achieves state-of-the-art zero-shot perfor-mance across both identically distributed and multiple out-of-distributionsettings, improving mIoU by up to 3.5 points over the strongest baselinewhile using over 7× fewer frames. The project homepage is available athttps://dart-vtg.github.io/.