EgoExoMoCap: Distributed Human Motion Capture via Ego- and Exocentric Body Tracking from Head-Mounted Devices

Human motion capture from head-mounted devices (HMDs)offers a scalable way to acquire real-world human motion and interac-tion data, which is crucial for applications in embodied AI and VR/AR.Existing approaches focus on either egocentric body tracking, estimat-ing the motion of the subject wearing the device, or exocentric track-ing, capturing the movements of people in the wearer’s surroundings. Sofar, these two paradigms have largely been explored in isolation. In thispaper, we propose a novel distributed framework that jointly leveragesego- and exocentric multi-modal signals for human motion estimationfrom HMDs. Unlike traditional motion capture systems requiring bulkymulti-camera setups or obtrusive mocap suits, our approach, EgoExo-MoCap, is as simple as two (or more) people, each wearing a pair ofsmart glasses. The method leverages head (plus potentially wrist) track-ing signals for accurate estimation of global motion in the 3D world andcombines context-aware image features based on DINOv3 to achieve ro-bustness in the presence of noise and occlusions. Extensive experimentson two in-the-wild datasets show that our approach can robustly recon-struct motion even in challenging scenarios.