Ex-Sim(3)-Reg: 2D-3D Correspondence Pruning via Extended Sim(3) Registration

Learning-based image-to-point-cloud (I2P) registration hasgarnered increasing attention in recent years. Nevertheless, existing meth-ods still struggle with severe outliers under challenging scenarios withunseen, low-inlier, or distorted cases. A fast and robust 2D-3D correspon-dence pruning method is therefore highly desirable. Recently, a promis-ing scheme lifts 2D-3D correspondences to 3D-3D correspondences usingdepth priors, casting correspondence pruning as a Sim(3) registrationproblem. However, depth priors estimated from monocular images areinherently noisy, which undermines the reliability of this scheme. In thispaper, to explicitly model non-negligible depth noise, we reformulatecorrespondence pruning as an extended Sim(3) registration problem andpropose a simple yet effective pruning algorithm termed Ex-Sim(3)-Reg.We further provide a theoretical analysis to justify the effectiveness ofour method. Extensive experiments on the 7-Scenes, RGBD-V2, Scan-Net, and TUM datasets demonstrate that Ex-Sim(3)-Reg achieves up to24.7% improvement in registration recall over state-of-the-art baselinemethods. Code is released at github.com/anpei96/ex-sim3-demo.