AnyMatch: Supercharging Universal Multi-Modal Image Matching with Large-Scale Single-View Images

Multi-modal image matching is essential for visual localiza-tion and multi-sensor fusion, but it is hindered by the scarcity of large-scale training data with precise geometric annotations. Existing real-world datasets suffer from prohibitive costs, limited scene diversity, anderrors in SfM-MVS pipelines, while synthetic methods struggle to main-tain 3D geometric consistency or achieve photorealistic appearance. Toaddress this, we propose AnyMatch, a novel framework that leveragesabundant, easily accessible single-view images at minimal cost to gen-erate rich multi-modal training data. AnyMatch integrates monoculardepth estimation, 3D reprojection, diffusion-based inpainting, and cross-modal image translation to synthesize multi-view, multi-modal imagepairs with 3D geometric fidelity. Crucially, our method provides annota-tions that strictly adhere to 3D geometric consistency through explicit3D reprojection, avoiding SfM-MVS error accumulation. Furthermore,AnyMatch offers strong scalability, enabling controllable scene diversityand annotation difficulty via adjustable input and camera parameters.We construct Any-syn, a large-scale synthetic multi-modal dataset us-ing AnyMatch. Experimental results show that matching networks (e.g.,LoFTR, EDM, RoMa) fine-tuned on Any-syn achieve substantial perfor-mance gains on multi-modal benchmarks, exhibiting superior generaliza-tion and robustness compared to models trained on existing data.