Decoupling Complexity from Scale in Latent Diffusion Model

Existing latent di!usion models typically couple scale withcontent complexity, using more latent tokens to represent higher-resolutionimages or higher–frame rate videos. However, the latent capacity re-quired to represent visual data primarily depends on content complex-ity, with scale serving only as an upper bound. Motivated by this obser-vation, we propose DCS-LDM, a novel paradigm for visual generationthat decouples information complexity from scale. DCS-LDM constructsa hierarchical, scale-independent latent space that models sample com-plexity through multi-level tokens and supports decoding to arbitraryresolutions and frame rates within a fixed latent representation. This la-tent space enables DCS-LDM to achieve a flexible computation–qualitytradeo!. Furthermore, by decomposing structural and detailed informa-tion across levels, DCS-LDM supports a progressive coarse-to-fine gen-eration paradigm. Experimental results show that DCS-LDM deliversperformance comparable to state-of-the-art methods while o!ering flex-ible generation across diverse scales and visual qualities.