Motion-aware Sparse Pipeline for Lightweight Object Tracking

Transformer-based object trackers are renowned for theirstrong performance, yet dense token processing often leads to prohibitivecomputational cost, limiting real-time deployment on edge devices. Whilerecent works explore token pruning to reduce computation, they oftenstop short of an end-to-end sparse pipeline, as early-layer token scores canbe noisy without a motion prior, and many trackers ultimately fall back todense reshaping to feed the dense prediction head that partially negatesthe savings. We introduce Motion-aware Sparse Tracker (MaST), a sparsetracking framework that makes sparsity effective from tokens to boxes.First, MaST injects a lightweight motion prior to refine cross-attention-based importance scores, enabling earlier and more stable token reductionin the search region. Second, we introduce a natively sparse predictionhead that operates directly on the retained unstructured tokens witha score-first, regress-once design, eliminating dense padding/reshapingand reducing redundant computation. Extensive experiments on multiplebenchmarks demonstrate that MaST establishes new state of the artamong lightweight trackers, where MaST-tiny attains 63.8 AUC on LaSOTand 80.1 SUC on TrackingNet, surpassing the prior best AsymTrack-Sby +1.0 AUC and +2.2 SUC while running at 152 FPS on Jetson Nano,nearly twice as fast as AsymTrack-S at 88 FPS. Code is available atgithub.com/TsingWei/MaST.