SEM-ROVER: Semantic Voxel-Guided Diffusion for Large-Scale Driving Scene Generation

Scalable generation of outdoor driving scenes requires 3Drepresentations that remain consistent across multiple viewpoints andscale to large areas. Existing solutions either rely on image or video gen-erative models distilled to 3D space, harming the geometric coherenceand restricting the rendering to training views, or are limited to small-scale 3D scene or object-centric generation. In this work, we propose a3D generative framework based on Σ-Voxfield grid, a discrete representa-tion where each occupied voxel stores a fixed number of colorized surfacesamples. To generate this representation, we train a semantic-conditioneddiffusion model that operates on local voxel neighborhoods and uses 3Dpositional encodings to capture spatial structure. We scale to large scenesvia progressive spatial outpainting over overlapping regions. Finally, werender the generated Σ-Voxfield grid with a deferred rendering moduleto obtain photorealistic images, enabling large-scale multiview-consistent3D scene generation without per-scene optimization. Extensive experi-ments show that our approach can generate diverse large-scale urbanoutdoor scenes, renderable into photorealistic images with various sen-sor configurations and camera trajectories while maintaining moderatecomputation cost compared to existing approaches.