MuSViT: A Foundation Vision Model for Sheet Music Representation

Foundation models have transformed vision and languageprocessing by providing rich, reusable representations that transfer acrossdiverse tasks. Sheet music, as a visual encoding of musical language,lacks such a strong domain-specific backbone. We introduce MuSViT(Music Score Vision Transformer): the first foundation vision modelfor sheet music representation—a ViT encoder pre-trained via MaskedAutoencoders on 9.7 million pages from the International Music ScoreLibrary Project (IMSLP). To handle the complexity of real-world scores,we adopt a two-stage curriculum: a synthetic warm-up on typeset scoresfollowed by large-scale training on the full IMSLP corpus. We evaluateMuSViT on four downstream tasks—full-page and staff-level music scorerecognition, music symbol detection, and score difficulty classification—under two scenarios: linear probing (frozen encoder) and fine-tuning.Under linear probing, MuSViT consistently outperforms modern visionencoders, revealing that general-purpose representations, regardless ofscale, fall systematically short on the structured symbolic propertiesMuSViT PaliGemma 2 Qwen3-VL MuSViT SoTAMuSViT-Light Kosmos-2.5 DINOv3-7B MuSViT-LightFull-Page Music Full-Page MusicScore Recognition Score RecognitionScore Difficulty Staff-Level Music Score Difficulty Staff-Level MusicClassification Score Recognition Classification Score RecognitionMusic Symbol Music SymbolDetection DetectionFig. 2: MuSViT performance across four downstream tasks. Left: Linear probing(frozen encoder)—MuSViT (solid) consistently outperforms general-purpose vision en-coders (dashed), demonstrating superior representation quality. Right: Fine-tuning—MuSViT generally outperforms state-of-the-art methods (SoTA). Axes represent nor-malized performance on each task (higher is better); see Section 3 for detailed results.of musical notation. Under fine-tuning, MuSViT generally improvesupon task-specific state-of-the-art methods. An additional embedding-transcription consistency analysis reveals that MuSViT encodes sym-bolic musical structure directly in its representation space—unlike otherencoders, whose embeddings do not correlate with music notation con-tent. These results establish MuSViT as a foundation backbone for sheetmusic understanding.