Self-transcendence: Is External Feature Guidance Indispensable for Accelerating Diffusion Transformer Training?

Recent works such as REPA have shown that guiding dif-fusion models with external semantic features (e.g., DINO) can signifi-cantly accelerate the training of diffusion transformers (DiTs). However,the use of pretrained external features as guidance signals introducesadditional dependencies. We argue that DiTs actually have the powerto guide the training of themselves, and propose Self-Transcendence,an effective method that achieves fast convergence using internal featuresupervision only. The desired internal guidance features should meet tworequirements: structurally clean to help shallow blocks separate noisefrom signal, and semantically discriminative to help shallow layers learneffective representations. With this consideration, we first align the DiTfeatures with the clean VAE latent features, a native component of la-tent diffusion, for a short training phase (e.g., 40 epochs) to improvetheir structural representations, then apply the classifier-free guidanceto the intermediate features, enhancing their discriminative capabilityand semantic expressiveness. These enriched internal features, learnedentirely within the model, are used as supervision signals to guide a newDiT training from scratch. Compared to existing self-contained meth-ods, our approach achieves a significant performance boost. It can evensurpass REPA, which uses the external DINO features as guidance, inboth generation quality and convergence speed for both class-to-imageand text-to-image generation tasks. Codes and models can be found athttps://github.com/csslc/Self-Transcendence.