Track4World: Feedforward World-centric Dense 3D Tracking of All Pixels

Estimating the 3D trajectory of every pixel from a monocularvideo is crucial and promising for a comprehensive understanding of the3D dynamics of videos. Recent monocular 3D tracking works demon-strate impressive performance, but are limited to either tracking sparsepoints on the first frame or a slow optimization-based framework fordense tracking. In this paper, we propose a feedforward model, calledTrack4World, enabling an efficient holistic 3D tracking of every pixel inthe world-centric coordinate system. Built on the global 3D scene repre-sentation encoded by a VGGT-style ViT, Track4World applies a novel3D correlation scheme to simultaneously estimate the pixel-wise 2D and3D dense flow between arbitrary frame pairs. The estimated scene flow,along with the reconstructed 3D geometry, enables subsequent efficient3D tracking of every pixel of this video. Extensive experiments on multiplebenchmarks demonstrate that our approach consistently outperforms ex-isting methods in 2D/3D flow estimation and 3D tracking, highlighting itsrobustness and scalability for real-world 4D reconstruction tasks. Projectpage: https://jiah-cloud.github.io/Track4World.github.io/.