TMI: Text-to-Image Meets Image-to-Image for Complementary Data Synthesis to Boost Long-Tailed Instance Segmentation

Large-vocabulary instance segmentation is constrained bylong-tailed category distributions and fine-grained inter-class ambiguity.While data synthesis offers a promising alternative, current paradigmshave complementary limitations: text-to-image (T2I) methods inheritnoisy pseudo-labels and struggle on rare classes, whereas copy-pastemethods compromise contextual realism. To address these issues, we pro-pose a hybrid pipeline coupling T2I generation with context-aware image-to-image (I2I) editing. The T2I branch provides broad category and scenediversity, while a teacher-student scheme ensures label reliability by se-lectively retaining only prompt-specified categories. To strengthen su-pervision for rare classes, we introduce VRAIN (Verified Rare-classAugmentation via INstructed editing), a novel I2I editor. VRAIN insertshigh-confidence instances at semantically appropriate locations withinin-the-wild scenes, yielding semantically coherent and visually naturaledits that reduce domain gaps and enable targeted augmentation. Onthe LVIS benchmark, our method surpasses existing baselines, improv-ing overall AP by up to +4.0 points and rare-class AP by up to +9.5points, while scaling effectively with backbone capacity.