TSEmbed: Unlocking Task Scaling in Universal Multimodal Embeddings

Despite the exceptional reasoning capabilities of MultimodalLarge Language Models (MLLMs), their adaptation into universal em-bedding models is significantly impeded by task conflict. To addressthis, we propose TSEmbed, a universal multimodal embedding frameworkthat synergizes Mixture-of-Experts (MoE) with Low-Rank Adaptation(LoRA) to explicitly disentangle conflicting task objectives. Moreover,we introduce Expert-Aware Negative Sampling (EANS), a novel strategythat leverages expert routing distributions as an intrinsic proxy for se-mantic similarity. By dynamically prioritizing informative hard negativesthat share expert activation patterns with the query, EANS effectivelysharpens the model’s discriminative power and refines embedding bound-aries. To ensure training stability, we further devise a two-stage learningparadigm that solidifies expert specialization before optimizing repre-sentations via EANS. TSEmbed achieves state-of-the-art performance onboth the Massive Multimodal Embedding Benchmark (MMEB) and real-world industrial production datasets, laying a foundation for task-levelscaling in universal multimodal embeddings.