OpenPanoD: Aligning Multimodal Prompts and Spherical Representations for Open-Vocabulary Panoramic Detection

Panoramic object detection has made steady progress, butexisting methods remain largely closed-set and therefore cannot recog-nize novel categories in open environments. Extending open-vocabularydetection to panoramas is challenging because prompt-image alignmentis weakened by two factors: semantic ambiguity in language-only promptsand severe geometric distortion caused by equirectangular projection. Wepresent OpenPanoD, a framework for open-vocabulary panoramic de-tection that aligns multimodal prompts with geometry-aware sphericalrepresentations. On the prompt side, a multimodal prompt encoder com-bines language descriptions with visual exemplars to obtain category em-beddings that are both generalizable and visually specific. On the imageside, GeoFormer maps panoramic features onto a quasi-uniform spher-ical grid and performs spherical attention to reduce projection-induceddistortion and boundary discontinuity. The resulting prompt and imageembeddings are fused in a unified detection head for BFoV/RBFoV pre-diction. Experiments on 360-Indoor and PANDORA show that Open-PanoD consistently improves novel-category performance over strongopen-vocabulary and visual-prompt baselines. These results demonstratethe importance of jointly addressing semantic ambiguity and panoramicgeometry for open-vocabulary detection in 360-degree scenes.