MVPruner: Dynamic Token Pruning for Accelerating Multi-view Vision-Language Models in Autonomous Driving

Vision-Language Models (VLMs) improve generalization andinterpretability in autonomous driving but suffer from efficiency issuesdue to long visual token sequences, particularly in standard multi-viewsettings. Existing token pruning methods employ fixed pruning rate al-location and static importance metrics, ignoring dynamic inter-view im-portance differences and the evolving information importance during in-ference. Our analysis reveals that multi-view VLMs inherently encodetask-related view priors in deeper layers and exhibit dynamic informa-tion requirements. Motivated by these findings, we propose MVPruner,a two-stage adaptive token pruning method that aligns pruning behaviorwith the model’s dynamic information requirements. The first stage al-locates pruning budgets based on the information diversity of each view,and retains tokens with consistent contribution across stages, ensuringsemantic representational capacity. The second stage allocates budgetsand selects tokens guided by instruction text to guarantee task align-ment. Experimental results on four benchmarks demonstrate the supe-rior performance of our method. For example, DriveMM equipped withMVPruner achieves 87.3% reduction in FLOPs, 4.97× speedup in pre-filling phase while retaining 98.5% accuracy on DriveLM benchmark.