Online Reasoning Video Object Segmentation

Reasoning video object segmentation predicts pixel-level masksin videos from natural-language queries that may involve implicit andtemporally grounded references. However, existing methods are devel-oped and evaluated in an offline regime, where the entire video is availableat inference time and future frames can be exploited for retrospective dis-ambiguation, deviating from real-world deployments that require strictlycausal, frame-by-frame decisions. We study Online Reasoning Video Ob-ject Segmentation (ORVOS), where models must incrementally interpretqueries using only past and current frames without revisiting previouspredictions, while handling referent shifts as events unfold. To supportevaluation, we introduce ORVOSB, a benchmark with frame-level causalannotations and referent-shift labels, comprising 210 videos, 12,907 anno-tated frames, and 512 queries across five reasoning categories. We furtherpropose a baseline with continually-updated segmentation prompts anda structured temporal token reservoir for long-horizon reasoning underbounded computation. Experiments show that existing methods struggleunder strict causality and referent shifts, while our baseline establishesa strong foundation for future research.