Detect by Track: Making Detector-Free Matcher Trackable

Detector–free matchers (e.g., EDM, JamMa) offer a strong accuracy– latency trade-off for pairwise matching. However, their implicit keypoint selection—where keypoints emerge as a by-product of matching—often leads to multi-view inconsistency and fragmented tracks. This limitation hinders applications that require connected tracks, such as object tracking, SfM, and SLAM. Prior attempts to mitigate this issue rely on coarsely quantized matches combined with heavy post-processing for multiple images; thus, they are typically limited to batch-based SfM pipelines. We take a different approach and directly address multi-view inconsistency through a Detect-by-Tracking (DeT) mechanism. Given an image pair I , I , and a sub-pixel query q , DeT steers the local A B A similaritymatrixofadetector-freematchersothattherow-wisesoftmaxrepresents the correspondence probability of an arbitrary sub-pixel query q . This produces A connected tracks across views without heavy post-processing, while preserving the core strength of detector-free matching: implicit keypoint selection driven by matching. Moreover, DeT can be incorporated into off-the-shelf matchers without additional training. Experiments demonstrate that DeT maintains the base detector-free matcher’s matching accuracy while producing connected tracks.