EffiDINO: Task-Specific Model Pruning via Gram Anchoring Subspace Consistency

Pruning visual foundation models has attracted considerableattention. However, existing methods focus on rigid point-to-point tokenalignment on a single dataset for pruning, suffering from two limitations:i) robustness degradation, and ii) task-specificity deficiency. To addressthese limitations, we propose a task-specific pruning pipeline, namedCut-ViT. Specifically, we first construct gram anchoring matrices fromboth spatial and semantic perspectives, and perform the subspace de-composition to extract the corresponding subspace bases. Basis-agnosticand residual constraints are then adopted to align the gram subspaces be-tween the native and pruned DINOv3 models along spatial and channeldimensions, enabling subnetworks to inherit robust feature representa-tions of native DINOv3. Furthermore, we design spectral entropy adapta-tion, which quantifies the information density of feature manifolds alongspatial and channel dimensions, thereby adapting the pruning objectiveto specific downstream tasks. Experiments show that Cut-ViT requiresapproximately one minute on a single A100 GPU to obtain subnetworksat various sparsity levels, using only 20.9% of the time and 45.5% of theGPU memory compared with previous methods, while achieving SOTAperformance on six tasks across nine datasets.