3D-ReGen: A Unified 3D Geometry Regeneration Framework

We consider the problem of regenerating 3D objects from2D images and initial 3D shapes. Most 3D generators operate in a one-shot fashion, converting text or images to a 3D object with limited con-trollability. We introduce instead MeshReGen, a 3D regenerator that isconditioned on an initial 3D shape. This conceptually simple formula-tion allows us to support numerous useful tasks, including 3D enhance-ment, reconstruction, and editing. MeshReGen uses a new condition-ing mechanism based on VecSet, which allows the regenerator to up-date or improve the input geometry with consistent fine-grained details.MeshReGen learns a widely applicable regeneration prior from o!-the-shelf 3D datasets via self-supervised pretext tasks and augmentations,without additional annotations. We evaluate both the geometric consis-tency and fine-grained quality of MeshReGen, achieving state-of-the-artperformance in controllable 3D generation across several tasks.Query pointsConditional ImagesCond.(a) Architecture Latents !! PE❌…3D Coarse Cond.DINO V2 OutputDiT blocks⟳ ×#Zero-InitVAE Proj. VAEEncoder C DecoderCross attnSelf attnMLP… … …RefinedLatents…Random(b) Applications LatentsCoarse Mesh Enhancement Partial Point Cloud Reconstruction Masked Mesh EditingFig. 2: Overview. MeshReGen takes both 2D image and initial 3D geometry as input,enabling explicit control over global geometry (e.g., pose, coarse shape) while improvingfine-grained details. The 3D condition is encoded as VecSet latents (z L ) that compactlyrepresent the global geometry. After taking the sum with the positional embeddings,these conditioning and random latents are di!used by a DiT into enhanced latents andthen decoded into a complete, high-quality 3D shape.