DRS-VPT: Directly Re-localizing in Scenes using a Vision and Point Transformer

We present DRS-VPT, a feed-forward transformer archi-tecture for foundational image-to-scan registration. Given query imagesand a reference 3D point cloud, the model predicts the scan pose andpoint map alongside the poses and point maps of each camera, all ex-pressed in the first camera’s frame. It additionally predicts a coarse-to-fine pyramid of per-point and per-pixel features for direct reprojectivealignment of the scan to the first image. This formulation unifies down-stream tasks such as camera–LiDAR calibration in autonomous driv-ing and indoor camera-to-map relocalization. A single DRS-VPT modelachieves state-of-the-art performance for image-to-LiDAR registration inautonomous driving, competitive indoor relocalization without trainingmap-specific weights, and strong zero-shot transfer to unseen environ-ments. We also show qualitatively that the model learns complex scan-to-image projection properties such as occlusion of back-facing points.