OneWorld: Taming Scene Generation with 3D Unified Representation Autoencoder

Existing diffusion-based 3D scene generation methods pri-marily operate in 2D image/video latent spaces, which makes maintain-ing cross-view appearance and geometric consistency inherently chal-lenging. To bridge this gap, we present OneWorld, a framework thatperforms diffusion directly within a coherent 3D representation space.Central to our approach is the 3D Unified Representation Autoencoder(3D-URAE); it leverages pretrained 3D foundation models and augmentstheir geometry-centric nature by injecting appearance and distilling se-mantics into a unified 3D latent space. Furthermore, we introduce token-level Cross-View-Correspondence (CVC) consistency loss to explicitly en-force structural alignment across views, and propose Manifold-Drift Forc-ing (MDF) to mitigate train–inference exposure bias and shape a robust3D manifold by mixing drifted and original representations. Comprehen-sive experiments demonstrate that OneWorld generates high-quality 3Dscenes with superior cross-view consistency compared to state-of-the-art2D-based methods.