Music-to-Dance Generation via Atomic Movements

Music-driven dance generation aims to produce human mo-tion that is both rhythmically synchronized and semantically consis-tent with music. While recent neural approaches have achieved impres-sive visual realism, they typically model motion as a continuous signaland neglect its compositional nature, making generated dances struc-turally incoherent and difficult to control. In this work, we introducea structure-aware framework that models choreography as a sequenceof atomic movements—semantically interpretable motion events thatserve as the building blocks of dance. To construct this atomic move-ment vocabulary, we first segment large-scale dance data and clusterthem into atomic movement groups. We then employ a large languagemodel to semantically relabel and refine the clusters, yielding a set ofinterpretable and reusable atomic movements. Based on these atomicmovement annotations, we design a two-stage generation framework thatmirrors the human choreography process. In the atomic movement plan-ning stage, the model predicts the type, duration, and timing of atomicmovements conditioned on the input music, forming a symbolic danceallocation. In the completion stage, a transition-aware generator syn-thesizes smooth and stylistically coherent motion conditioned on theplanned structure. Extensive experiments demonstrate that our methodproduces dances with significantly improved structural coherence, rhyth-mic alignment, and perceptual naturalness compared to existing base-lines, while providing enhanced interpretability and controllable edit-ing through explicit structural representation. The code is available at(A) Real music and choreography exhibits highly structural formation.Repeated patterns Repeated patternsSimilar or symmetrical atomic movements(B) Existing end-to-end works overlook such structural organization.Fig. 1: (A) There exists explicit repeated patterns in music and choreography. (B)Existing methods, usually in an end-to-end form, treat music-to-dance generation as asimple sequence-to-sequence task, neglecting the inner structure.