Histopathology Multi-modal Embedding for Pathology Composed Retrieval

To overcome the black-box nature of predictive AI and thehallucination risks of generative models, retrieval-based models o!er aninterpretable, evidence-based paradigm for pathology clinical workflow.However, real-world clinical queries are inherently interleaved (e.g., pathol-ogy images and text). Current dual-encoders su!er from an Archi-tectural Mismatch, lacking the mechanism to fuse such composedqueries. To address this, we formalize the task of Pathology ComposedRetrieval (PCR). While Multimodal Large Language Models (MLLMs)o!er deep-fusion capabilities, directly applying them exposes a TaskMismatch and a Domain Mismatch. To resolve these challenges,we propose HOMIE, a model-agnostic adaptation framework that trans-forms any generative MLLM into a specialized pathology retrieval expert.Evaluated on our newly introduced PCR Benchmark, a lightweight 2B-parameter HOMIE variant substantially outperforms existing paradigms,surpassing specialized 7B pathology MLLMs and dual-encoders by largemargins on composed retrieval, while maintaining strong performanceon traditional simple retrieval. The project page is available at https://qfchou.github.io/HOMIE_page/.