MomentSeg: Moment-Centric Sampling for Enhanced Referring Video Object Segmentation

Referring Video Object Segmentation (RefVOS) seeks to seg-ment target objects in videos guided by natural language descriptions,demanding both temporal reasoning and fine-grained visual comprehen-sion. Existing sampling strategies for LLM-based approaches typicallyrely on either handcrafted heuristics or external keyframe models. Theformer often overlooks essential temporal cues, while the latter increasessystem complexity. To address this, we propose a unified frameworkthat jointly optimizes Temporal Sentence Grounding (TSG) and Re-fVOS, naturally incorporating key moment grounding capability. Duringtraining, we introduce a novel TSG paradigm that employs a dedicated[FIND] token for key moment identification through temporal token sim-ilarity matching, thereby avoiding the need for external timestamp en-codings. For inference, we design a Moment-Centric Sampling (MCS)strategy that densely samples informative moments while sparsely sam-pling non-essential frames, preserving both motion details and globalcontext. To further enhance tracking stability, we develop BidirectionalAnchor-updated Propagation (BAP), which leverages the most relevantmoment as start point for high-quality mask initialization and dynam-ically updates at sampled points to mitigate accumulated errors. Codewill be released at https://github.com/Dmmm1997/MomentSeg.