Dynamic World Generation Made Efficient

Generating large dynamic 3D worlds is important for appli-cations such as virtual reality, simulation, and robotics. However, the lackof large-scale 3D dynamic training data prevents training world-scale 3Dgenerative models directly. Recent progress in video generation providespowerful priors for generating 3D dynamic visual content. As a result,current approaches typically generate videos from multiple camera viewsusing image or video diffusion models and fuse them into a 3D represen-tation. However, these models generate each view independently, makingit difficult to maintain motion consistency across viewpoints. Existing so-lutions address this problem by sequentially generating each view whileupdating an explicit 3D representation, which introduces a significantcomputational bottleneck. In this work we propose D YN W ORLD, a frame-work that enables parallel generation of motion-consistent videos acrossmultiple views. Our key insight is that the latent tokens of transformer-based video generators implicitly encode spatial and dynamic informa-tion that can be geometrically related across cameras. We thus intro-duce a Spatial Token Memory that stores latent video tokens along withtheir estimated 3D positions. During generation, tokens from one vieware back-projected into 3D and re-projected into other views, allowingmultiple video generators to share consistent motion context through at-tention. This mechanism enforces cross-view motion consistency whileallowing all views to be generated simultaneously. Experiments showthat our method produces large, diverse, view-consistent dynamic worldswhile reducing generation time by an order of magnitude compared tosequential pipelines.