Not All Prediction Targets Keep Training-Free Diffusion Guidance on the Manifold

Training-free guidance (TFG) steers a pretrained diffusionmodel toward a desired attribute at inference. To be effective, this guid-ance must be applied from the earliest, high-noise steps of sampling.Because its objective (a classifier or energy) is defined on clean images,ϵ- and v-prediction models must first estimate the clean image x̂ fromthe noisy state at each step, and the accuracy of that estimate deter-mines how easily guidance drifts off the data manifold. x-prediction,a recent alternative, outputs the clean image directly, removing thissource of error even at high noise. This is our motivation. We providea theoretical analysis of how each prediction target shapes this accu-racy, and introduce guided-class FID (Child FID), a metric that exposesthe manifold damage standard evaluation misses. Experiments on anew fine-grained bird benchmark and on style transfer confirm that x-prediction keeps guided samples on the manifold most reliably, making itthe strongest foundation for training-free guidance. Code is available athttps://github.com/ManLuML/on-manifold-tfg.