P-CORE: Self-Supervised Surface Consistency for Point-Based Neural Editing

Advances in neural rendering have enabled high-fidelity multi-view reconstruction of 3D scenes. However, free-form non-rigid shapeediting remains a significant challenge. Point-based neural representationsare highly desirable for multi-view reconstruction because they lack fixedconnectivity, which does not constrain the learned surface topology tothat of the initialization. Yet this same property causes point-based rep-resentations to struggle with holes and surface discontinuities under largedeformations. To address this, we propose a novel self-supervised methodto enable point-based representations to adapt to large deformations with-out requiring ground truth multi-view images of deformed geometry. Thekey idea is to generate random deformations and to ensure consistencyin the predicted surface before and after deformation. In particular, thesurface prediction from the deformed point cloud should be the sameas the deformation applied to the surface prediction from the originalpoint cloud. We incorporate our approach into attention-based pointrepresentations, which differ from splatting-based point representationsin their use of a learned interpolation kernel between points as opposedto a Gaussian kernel around each point. This learned interpolation kernelcan learn to adapt to large deformations, without requiring addition orremoval of points. We show that our framework significantly enhancesits robustness to large deformations. Experiments on synthetic geometryediting benchmarks (Neural Editor, Objaverse) demonstrate that ourapproach outperforms existing point-based methods in zero-shot editingand significantly reduces artifacts. Furthermore, qualitative results on theDTU and Mip-NeRF 360 datasets demonstrate our method’s effectivenesson real-world scenes.