ReViV: Reconstructing the Viewer and the View in 4D from Monocular Egocentric Video

Egocentric devices, such as wearable front-facing cameras,provide a unique perspective for capturing the continuous interactionbetween a human viewer and the surrounding environment. A holisticand efficient multimodal model capable of reconstructing this 4D rep-resentation is therefore highly desirable. However, existing approachesoften rely on auxiliary inputs such as pre-computed camera trajecto-ries, treat scene perception and human ego-motion modeling as separateproblems despite their strong interdependency, and suffer from slow in-ference time. To address these limitations, we present ReViV, the firstunified framework for holistic egocentric 4D reconstruction that extractsboth viewer and view dynamics from a single monocular RGB video.We formulate the task as learning the full joint probability distribu-tion over multimodal signals, including RGB video, camera trajectory,gaze direction, full-body motion, hand motion, and depth. Powered bya Masked Generative Egocentric Transformer, ReViV operates within asingle feed-forward architecture to simultaneously reconstruct the tem-porally consistent 4D reconstruction across the viewer and the view withfast inference speed. Extensive experiments on diverse benchmarks, in-cluding HoloAssist, HOT3D, ARCTIC, Aria Digital Twin, and TACO,demonstrate that ReViV achieves state-of-the-art accuracy and efficiencyacross holistic ego-body, hand, and gaze reconstruction, camera track-ing, while maintaining highly competitive egocentric depth estimation,without relying on heavy task-specific priors. Code and models are fullyopen-sourced: https://reviv4d.github.io/.