VisCoP: Visual Probing for Video Domain Adaptation of Vision Language Models

Large Vision Language Models (VLMs) excel at general vi-sual reasoning tasks, but their performance degrades sharply when de-ployed in novel domains with substantial distribution shifts compared towhat was seen during pretraining. Existing approaches to adapt VLMsto novel target domains rely on finetuning standard VLM components.Depending on which components are finetuned, these approaches eitherlimit the VLMs ability to learn domain-specific features, or lead to catas-trophic forgetting of pre-existing capabilities. To address this, we intro-duce Vision Contextualized Probing (VisCoP), which augments theVLM’s vision encoder with a compact set of learnable visual probes,enabling domain-specific features to be learned with only minimal up-dates to the pretrained VLM components. We evaluate VisCoP acrossthree challenging domain adaptation scenarios: cross-view (exocentric →egocentric), cross-modal (RGB → depth), and cross-task (human under-standing → robot control). Our experiments demonstrate that VisCoPconsistently outperforms existing domain adaptation strategies, achiev-ing superior performance on the target domain, while better retainingcapabilities from the source domain. We will release all code, models,and evaluation protocols to facilitate future research in this direction.