PixelU: A U-Shaped Transformer for Efficient End-to-End Pixel Diffusion

End-to-end pixel-space diffusion models bypass the lossy com-pression of Latent Diffusion Models (LDMs) but struggle to jointly modellow-frequency semantics and high-frequency signals in high-dimensionalspace. Existing works heavily rely on complex pixel decoders to alleviatethis issue. In this paper, we challenge this trend by revealing that thesedecoders primarily compensate for the optimization difficulties inherentto velocity prediction (v-prediction). Under the clean data paradigm (x-prediction), they are redundant. Motivated by this insight, we advocatefor simplicity over complexity and introduce PixelU, a minimalist, single-stage U-shaped Diffusion Transformer tailored for pixel space. PixelUabandons auxiliary decoders in favor of zero-cost skip connections, whichprovide an “information highway” that directly routes uncorrupted high-frequency spatial details from shallow to deep layers. To further enablethe backbone to focus exclusively on modeling low-frequency semantics,we introduce a constant-channel spatial down-sampling mechanism asa natural low-pass filter, which compresses deep features into a com-pact, low-frequency semantic manifold. Extensive experiments demon-strate that this decoupling of frequencies could outperform the strongbaseline (JiT-G) with only about 1/3 of its computation cost. On Im-ageNet 256 × 256 and 512 × 512, PixelU achieves FID of 1.63 and 1.92respectively, surpassing recent pixel-space methods and establishing asimple yet powerful new paradigm for end-to-end diffusion models. Codewill be available at https://github.com/gzp6688/PixelU.