One4D: Unified 4D Generation and Reconstruction via Decoupled LoRA Control

We present One4D, a unified framework for 4D generationand reconstruction that produces dynamic 4D content as synchronizedRGB frames and pointmaps. By consistently handling varying sparsitiesof conditioning frames through a Unified Masked Conditioning (UMC)mechanism, One4D can seamlessly transition between 4D generationfrom a single image, 4D reconstruction from a full video, and mixed gen-eration and reconstruction from sparse frames. Our framework adapts apowerful video generation model for joint RGB and pointmap generation,with carefully designed network architectures. The commonly used diffu-sion finetuning strategies for depthmap or pointmap reconstruction oftenfail on joint RGB and pointmap generation, quickly degrading the basevideo model. To address this challenge, we introduce Decoupled LoRAControl (DLC), which employs two modality-specific LoRA adapters toform decoupled computation branches for RGB frames and pointmaps,connected by lightweight, zero-initialized control links that graduallylearn mutual pixel-level consistency. Trained on a mixture of syntheticand real 4D datasets under modest computational budgets, One4D pro-duces high-quality RGB frames and accurate pointmaps across both gen-eration and reconstruction tasks. This work represents a step towardgeneral, high-quality geometry-based 4D world modeling using video dif-fusion models. Project page: https://mizhenxing.github.io/One4D.