Leveraging Dark Knowledge for Intrinsic Multimodal Out-of-Distribution Detection

Out-of-distribution (OOD) detection is crucial for the safedeployment of deep neural models in applications such as autonomousdriving. With the emerging multimodal nature of modern applications,recent attention has shifted toward OOD detection in multimodal set-tings. However, current multimodal OOD detection methods fail to fullyexploit the synergy among modalities: they treat all modalities equally,disregarding their varying detection performance, and they are unableto capture the diverse uncertainty information encoded at the logit level.In this paper, we propose to exploit the dark knowledge within unimodalexperts as the key to revealing their synergy. To this end, we introducea self multimodal OOD distillation framework, which leverages logits asuncertainty-aware soft targets to train a holistic model that operates inthe joint embedding space of all modalities. Specifically, the proposedframework accounts for the negative effects of underperforming modal-ities and effectively fuses both the rich feature-level knowledge and thelogit-level knowledge of modalities. As a result, our method improvesthe performance of current state-of-the-art multimodal OOD detectionmethods, achieving gains of up to 30% across diverse OOD detectionbenchmarks, spanning two tasks and five multimodal OOD datasets.