Accelerating Diffusion Transformers with Gaussian Process Rectified Feature Cache

Diffusion Transformers have become the dominant paradigmin generative AI, but their high computational costs severely hinderreal-time applications. Prediction-based feature caching is widely usedto accelerate diffusion transformers; however, as the number of stepsincreases, the deviation between its predictions and the reference full-compute trajectory gradually grows. An intuitive idea is to use an onlineregression model to dynamically correct this deviation, but it faces theissue of label data being unavailable during the acceleration process.This paper presents a statistical observation that the residuals betweenthe features of full computation steps using caching methods and refer-ence full-compute trajectory locally exhibit a zero-mean Gaussian dis-tribution. By treating the features of full computation steps as noisyobservations of reference features, the data acquisition problem is re-solved. Based on this observation, a plug-and-play GP-Refiner correctionframework is proposed. This method utilizes Gaussian Process Regres-sion for correction and, leveraging the properties of GPR, introducesan uncertainty-adaptive computation strategy that triggers necessaryfull-computation calibration by monitoring the posterior variance in realtime. Experiments demonstrate significant improvements across differentmodels when combined with various state-of-the-art methods. Integrat-ing the proposed framework with TaylorSeer reduces the computationalload by 19.3% while improving PSNR by 0.9 dB and reducing LPIPSfrom 0.46 to 0.29. Code is available in https://github.com/Aredstone/GP-Refiner.