Following Motion for Sequential Modeling in Video Frame Interpolation
Abstract
State Space Models (SSMs) have surfaced as a promisingarchitecture in Video Frame Interpolation (VFI), as they can capturelong-range dependencies with linear computational complexity. However,their predefined scanning order limits their effectiveness in modelingthe dynamic motion trajectories inherent in VFI problems. To tacklethis challenge, we propose Motion-Guided Mamba for Video FrameInterpolation (MGMVFI), an adaptation of the selective state spacemodel tailored explicitly for VFI. MGMVFI introduces Motion-GuidedSerialization (MGS), which leverages optical flow to define a motion-adaptive 1D input order for the SSM. This aligns the causal state up-dates with semantically related tokens, enabling motion-consistent fea-ture propagation, particularly for large and dynamic motions. Addition-ally, to mitigate the unreliable feature representations caused by inaccu-rate optical flow estimates, we introduce contextual synthesis that uti-lizes the surrounding spatial context for robust inter-frame feature syn-thesis. These components are seamlessly integrated within our tailoredMamba architecture, which also employs a lightweight refinement blockto enhance local detail reconstruction at a reduced computational cost.Extensive experiments on standard VFI benchmarks demonstrate thatMGMVFI achieves state-of-the-art performance, particularly on complexand dynamic motions, thereby establishing a new direction for sequencemodeling in video interpolation.