LightFusion: A Light-weighted, Double Fusion Framework for Unified Multimodal Understanding and Generation

Unified multimodal models have recently shown remarkablegains in both capability and versatility, yet most leading systems are stilltrained from scratch and require substantial computational resources. Inthis paper, we show that competitive performance can be obtained farmore efficiently by strategically fusing publicly available models special-ized for either generation or understanding. Our key design is to re-tain the original blocks while additionally interleaving multimodal self-attention blocks throughout the networks. This double fusion mechanism(1) effectively enables rich multi-modal fusion while largely preservingthe original strengths of the base models, and (2) catalyzes synergisticfusion of high-level semantic representations from the understanding en-coder with low-level spatial signals from the generation encoder. By train-ing with only ∼35B tokens, this approach achieves strong results acrossmultiple benchmarks: 0.91 on GenEval for compositional text-to-imagegeneration, 82.16 on DPG-Bench for complex text-to-image generation,6.06 on GEditBench, and 3.77 on ImgEdit-Bench for image editing. Wewill fully release the entire suite of code, model weights, and datasets tosupport future research on unified multimodal modeling.