Infinite-Homography as Robust Conditioning for Camera-Controlled Video Generation

Recent progress in video diffusion models has spurred grow-ing interest in camera-controlled novel-view video generation for dy-namic scenes, aiming to provide creators with cinematic camera con-trol capabilities in post-production. A key challenge in camera-controlledvideo generation is ensuring fidelity to the specified camera pose, whilemaintaining view consistency and reasoning about occluded geometryfrom limited observations. To address this, existing methods either traintrajectory-conditioned video generation model on trajectory–video pairdataset, or estimate depth from the input video to reproject it alonga target trajectory and generate the unprojected regions. Nevertheless,existing methods struggle to generate camera-pose–faithful, high-qualityvideos for two main reasons: (1) reprojection-based approaches are highlysusceptible to errors caused by inaccurate depth estimation; and (2)the limited diversity of camera trajectories in existing datasets restrictslearned models. To address these limitations, we present InfCam, a depth-free, camera-controlled video-to-video generation framework with highpose fidelity. The framework integrates two key components: (1) infi-nite homography warping, which encodes 3D camera rotations directlywithin the 2D latent space of a video diffusion model. Conditioningon this noise-free rotational information, the residual parallax term ispredicted through end-to-end training to achieve high camera-pose fi-delity; and (2) a data augmentation pipeline that transforms existing syn-thetic multiview datasets into sequences with diverse trajectories and fo-cal lengths. Experimental results demonstrate that InfCam outperformsbaseline methods in camera-pose accuracy and visual fidelity, generaliz-ing well from synthetic to real-world data.