GTR: Guide-Then-Refine Token Compression for Training-Free Acceleration of Video-LLMs

Video large language models (Video-LLMs) have achieved re-markable progress in video understanding tasks, but their heavy compu-tational overhead during inference severely limits deployment in resource-constrained scenarios. This highlights the urgent need for efficient infer-ence acceleration frameworks tailored to Video-LLMs. Existing accelera-tion methods typically apply a homogeneous processing approach to allvideo frames, disregarding the fact that each frame within a video clipconveys distinct visual information. Additionally, many of them under-utilize text queries, or add text guidance only through costly pre-trainingor intrusive LLM modifications, underexploiting the text-video align-ment that is critical in long-form video understanding where redundantvisual content is common. To address this limitation, we propose Guide-Then-Refine (GTR), a plug-and-play inference acceleration frameworkfor Video-LLMs that realizes token compression through a two-stagemechanism. First, for all tokens in each video frame, we compute bothglobal scores and text-aware scores. Based on the sum of these scores forall tokens in a frame, we dynamically adjust the token retention ratio foreach frame. Subsequently, according to the retention ratio, we calculatethe local score of each token to evaluate its importance within the currentframe, thereby selecting critical visual tokens. Extensive evaluations onmainstream video benchmarks and various Video-LLMs demonstrate theeffectiveness of our approach. With only 15% of visual tokens retained,our method maintains an average of 95.8% of the original performanceacross four benchmarks.