GroundUp: Rapid Sketch-Based 3D City Massing

We propose GroundUp, the first sketch-based ideation tool for 3D city massing of urban areas. Architects start by making designs that create balance between constructed masses and open spaces, but existing software is aimed at late-stage precise building details. GroundUp aims to allow architects to easily iterate earlier, as they try out and share their ideas, moving back and forth between 2D sketching and 3D. Inspired by feedback from architects and existing workflows, our user sketches the initial footprints of multiple buildings in a top-down view. Our proposed interface then enables them to preview and sketch in a second, perspective view. All the while, they can visualize and navigate a 3D model, inferred to match the two sketches, leading them to change or refine the designs. The model driving our interface has two main components. First, we propose a tailored depth prediction network for perspective sketches, which exploits the top-down view cues. Second, we integrate a state-of-the-art latent diffusion model to enable the generation of plausible-looking buildings. This diffusion model uses a heightfield representation, which allows us to construct the city ''from the ground up''. It is conditioned on the top-down sketch and partial depth cues derived from the perspective sketch. We will release the code, datasets, and interface upon acceptance.