Segmentation-Guided Homography Estimation for Long-Term Planar Tracking

Recent state-of-the-art visual trackers produce high qualityand long-term-stable segmentation masks. We propose to leverage thesestrengths for planar object tracking, in which the goal is to estimate aprecise 8-degrees-of-freedom homography pose, a geometric representationnot estimated by segmentation trackers. We present SAM-H – a planarobject tracker that estimates homographies from segmentation mask con-tours via a training-free pipeline. When SAM-H is applied to masks fromSAM 2 [21], it sets a new state-of-the-art performance on the challengingPlanarTrack [15] benchmark by a large margin, +18.4pp on the p@5metric. We further show that segmentation-based and correspondence-based homography estimation are complementary, and propose WOFT-SAM, which out-performs all prior methods on both PlanarTrack [15]and POT-210 [14]. We also provide precise re-annotations of Planar-Track initial poses, enabling more accurate benchmarking in the high-precision p@5 metric. The code and the re-annotations are available athttps://github.com/serycjon/WOFTSAM.