On the Diffusibility of High-Dimensional Latents

Representation Autoencoders (RAEs) enable diffusion mod-els to operate in the feature spaces of pretrained visual encoders. How-ever, many off-the-shelf encoders are not optimized for faithful recon-struction and often discard fine-grained visual details. Finetuning theseencoders for image reconstruction can recover such details, but we showthat it also reduces the effective dimensionality of the resulting represen-tation space. We analyze how this altered geometry affects generation inhigh-dimensional feature spaces. Under this geometry, standard veloc-ity prediction in flow matching can require the model to fit orthogonalnoise directions outside the low-dimensional signal manifold, making op-timization inefficient. This motivates the clean data parameterization(x0 -prediction), which focuses learning on the underlying signal mani-fold. Across experiments with multiple reconstruction-finetuned featuresets, we show that x0 -prediction consistently improves text-to-image gen-eration performance.