OmniMamba: Efficient and Unified Multimodal Understanding and Generation via State Space Models

Recent advancements in unified multimodal understandingand visual generation (or multimodal generation) models have been hin-dered by their quadratic computational complexity and dependence onlarge-scale training data. We present OmniMamba, the first linear-architecture-based multimodal generation model that generates both text and imagesthrough a unified next-token prediction paradigm. The model fully lever-ages Mamba-2’s high computational and memory efficiency, extending itscapabilities from text generation to multimodal generation. To addressthe data inefficiency of existing unified models, we propose two key inno-vations: (1) decoupled vocabularies to guide modality-specific generation,and (2) task-specific LoRA for parameter-efficient adaptation. Further-more, we introduce a decoupled two-stage training strategy to mitigatedata imbalance between two tasks. Equipped with these techniques, Om-niMamba achieves competitive performance with JanusFlow while sur-passing Show-o across benchmarks, despite being trained on merely 2Mimage-text pairs, which is 1,000 times fewer than Show-o. Notably, Om-niMamba stands out with outstanding inference efficiency, achieving upto a 119.2× speedup and 63% GPU memory reduction for long-sequencegeneration compared to Transformer-based counterparts. Code and mod-els are released at https://github.com/hustvl/OmniMamba.