Enhancing Interpretability in CLIP with Optimal Transport-based Submodular Optimization for Ophthalmic Imaging

Although Contrastive Language-Image Pretraining (CLIP)has shown promising transfer ability for ophthalmic image analysis, itsopaque decision-making process hinders interpretability, especially inophthalmic settings where probability-only outputs may be difficult forclinicians and patients to trust. Providing visual evidence for predictionscan therefore make models more inspectable and interpretable. How-ever, existing explainability methods often struggle to produce spatiallyconsistent and semantically aligned heatmaps that highlight prediction-relevant anatomical or pathological evidence. To bridge this gap, wepropose OTSMDL, a post-hoc framework for interpretable region selec-tion that couples submodular-inspired scoring with unbalanced optimaltransport. We formulate the identification of salient evidence regions as asubset selection problem. Specifically, submodular-inspired utilities firstconstruct an initial candidate subset and a score-guided source distri-bution, which are then used to initialize the Optimal Transport (OT)process. Subsequently, OT performs global transport-based re-rankingover all candidates, constructing a unified cost matrix to encourage cross-region competition and align candidate superpixels with text prompts.We evaluate OTSMDL on seven ophthalmic datasets spanning color fun-dus photography, OCT, and glaucoma assessment, with additional Ima-geNet experiments for out-of-domain analysis. Under standard post-hocDeletion/Insertion protocols and ground-truth mask localization met-rics, OTSMDL generally outperforms baselines in region-level evidencegrounding, achieving top-1/2 performance in most evaluation settings.