World Reconstruction From Inconsistent Views

Video diffusion models generate high-quality and diverse worlds;however, individual frames often lack 3D consistency across the outputsequence, which makes the reconstruction of 3D worlds difficult. To thisend, we propose a new method that handles these inconsistencies bynon-rigidly aligning the video frames into a globally-consistent coordi-nate frame that produces sharp and detailed pointcloud reconstructions.First, a geometric foundation model lifts each frame into a pixel-wise 3Dpointcloud, which contains unaligned surfaces due to these inconsisten-cies. We then propose a tailored non-rigid iterative frame-to-model ICPto obtain an initial alignment across all frames, followed by a global opti-mization that further sharpens the pointcloud. Finally, we leverage thispointcloud as initialization for 3D reconstruction and propose a novelinverse deformation rendering loss to create high quality and explorable3D environments from inconsistent views. We demonstrate that our 3Dscenes achieve higher quality than baselines, effectively turning videomodels into 3D-consistent world generators.