SDSA: Shallow-Deep Squeezing Adapter for Vision-Language Models

Recent adapters for vision-language models (VLMs) still suf-fer from dense cross-modal interactions and limited control over align-ment capacity. To address this issue, we propose a two-stage Shallow-Deep Squeezing Adapter (SDSA) that explicitly regulates cross-modalinteraction density and alignment capacity. In the shallow squeezingstage, SDSA leverages token-level masking to impose structured spar-sity within a sparse bottleneck space. Then, in the deep squeezing stage,SDSA applies a shared low-rank transformation and a cross-modal atten-tion module: the low-rank module consolidates alignment into a compactset of dominant shared directions, while cross-modal attention refinesrepresentations through selective interaction. Extensive experiments on11 datasets show that SDSA delivers superior base-to-novel generaliza-tion and cross-dataset evaluation, and significantly improves the gener-alizability of VLMs under few-shot conditions. The code can be found athttps://github.com/haowang-ac/SDSA.