CrossFeat: Bridging Imaging Modalities in Feature Descriptor Space

Most advances in keypoint descriptions address monomodalsettings, where image variations arise from viewpoint, illumination, orcontrast changes. Multimodal scenarios involve images produced by fun-damentally different sensing processes, such as multispectral imaging,RGB-depth, satellite imagery, or medical imaging, causing the samestructures to appear differently. A common solution to cross-modal de-scription is to train descriptors for each modality pair, which requires re-training whenever the modalities change, or to train large models, whichincur a significant increase in runtime. Instead, we propose CrossFeat,a framework that enables an existing monomodal descriptor to operateacross modalities. Our method learns a crossing function in descriptorspace that maps features from one modality to a representation com-patible with another. To preserve the structural information capturedby the original descriptor, CrossFeat introduces a geometry–appearancedisentanglement such that only appearance is altered while the geomet-ric properties are preserved. Experiments across multiple domains anddatasets demonstrate improved performance in multimodal matching.Code available here: https://github.com/paulschneider01/CrossFeat