Skip to yearly menu bar Skip to main content


Poster

IAM-VFI : Interpolate Any Motion for Video Frame Interpolation with motion complexity map

Kihwan Yoon · Yong Han Kim · Sungjei Kim · Jinwoo Jeong

Strong blind review: This paper was not made available on public preprint services during the review process Strong Double Blind
[ ]
Fri 4 Oct 1:30 a.m. PDT — 3:30 a.m. PDT

Abstract:

Within the video, different regions have varying motion complexity, with simple regions containing static or global motion and complex regions containing fast motion or lots of local motion. In recent years, the performance of flow-based Video Frame Interpolation (VFI) algorithms has improved significantly. However, existing training methods train on randomly cropped regions of train data without considering the complexity of the motion. As a result, they cannot handle all regions of the frame that contain varying motion complexity. To solve this problem, we propose a novel VFI approach (IAM-VFI) that can interpolate any motion by considering the motion complexity of all regions in the frame. First, we propose a training data classification method for motion optimization based on each motion complexity. Then, using the proposed data, a flow estimation network generates optimized results for each complexity. Finally, we propose a Motion Complexity Estimation Network (MCENet) to generate a Motion Complexity Map (MCM) that can estimate the motion complexity of each region. Our proposed methods can be easily applied to most flow-based VFI algorithms. Experimental results show that the proposed method can interpolate any motion and significantly improve the performance of existing VFI algorithms.

Live content is unavailable. Log in and register to view live content