CURE: Cumulative Knowledge Reuse for Efficient Device-Server Hybrid Inference in Vision-Language Models

On-device vision-language models (VLMs) deployed on edgedevices often encounter distribution shifts between pre-training data andtest data, which can significantly degrade their performance. To miti-gate this degradation, device-server hybrid inference allows an on-deviceVLM to delegate uncertain test inputs to a more powerful server-sideVLM. However, frequently delegating to the remote server at test timeinflates data-transfer costs and server-call latency, limiting the practi-cality of hybrid inference in real-world applications. Thus, to reduce thedelegation frequency without sacrificing the benefits of server assistance,we present CURE, a cumulative knowledge reuse framework for device-server hybrid inference, which efficiently enhances on-device VLMs byaccumulating and reusing the server-side VLM’s knowledge. As a result,CURE enables the on-device VLM to approximate server-level predic-tions with far fewer delegations, thereby amortizing both data-transfercosts and server-call latency across the test stream. Extensive experi-ments across 13 datasets show that CURE closely matches the accuracyof an always-delegate-to-server baseline, reduces delegation frequency by66.74% on average, and improves end-to-end runtime by up to 3.67×,demonstrating the practicality of CURE for real-world deployment.