Task-driven Processing with Coarse-to-Fine Glimpse-based Active Perception

State-of-the-art vision models process images in their en-tirety, lacking the ability to selectively zoom in on relevant regions. Thislimitation is particularly acute in scenarios where processing must beconditioned on a specific task – such as instance detection, which re-quires localizing a specific object in a high-resolution, cluttered scene. Insuch settings, critical details are easily lost as images are often resizedto match the model dimensions and computational constraints. We in-troduce Coarse-to-Fine Glimpse-based Active Perception (CF-GAP), atask-driven front-end that enhances high-resolution processing of exist-ing instance detectors. CF-GAP selectively directs a sequence of limited-view glimpses across the scene, utilizing task information to iterativelyrefine focus on the most relevant regions. These localized regions arethen processed at high resolution by a downstream instance detector.By avoiding full-image processing and eliminating irrelevant confound-ing information, CF-GAP improves Average Precision (AP) by up to20% across various state-of-the-art instance detectors on the HR-InsDetand Robotools benchmarks, while further enabling lightweight detectorsto outperform their larger counterparts.