Bridging Vision and Language Concepts through Optimal Transport Semantic Flow

Concept Bottleneck Models (CBMs) promise transparent rea-soning by predicting through human-interpretable concepts, yet theireffectiveness fundamentally depends on how well visual and textual rep-resentations are aligned or matched. Existing vision–language CBMs of-ten rely on pre-aligned encoders or global cosine similarity, which ob-scures fine-grained concept localization and fails to reflect true seman-tic geometry. In this work, we rethink concept alignment as dynamiccross-modal transport process instead of static projection and proposeOptimal Transport Flow Concept Bottleneck Model (OTF-CBM). Itfirst learns a data-driven semantic cost via Inverse Optimal Transport tomeasure cross-modal distances, and then performs unbalanced optimal-transport-based flow matching to model semantic transitions betweenvisual patches and textual concepts. With velocity-based concept activa-tion, OTF-CBM captures interpretable geometric relations without ODEintegration. Experiments further show that OTF-CBM achieves greatclassification accuracy and concept faithfulness, offering a new geomet-ric and dynamical perspective for interpretable cross-modal reasoning.Our code can be found at github.com/ChenyangZhang00/OTF-CBM.