DiT as Real-Time Rerenderer: Streaming Video Stylization with Autoregressive Diffusion Transformer

Recent advances in video generation models have signifi-cantly accelerated video generation and related downstream tasks. Amongthese, video stylization holds important research value in areas suchas immersive applications and artistic creation, attracting widespreadattention. However, existing diffusion-based video stylization methodsstruggle to maintain stability and consistency when processing long videos,and their high computational cost and multi-step denoising make themdifficult to apply in practical scenarios. In this work, we propose RTR-DiT (DiT as Real-Time Rerenderer), a streaming video stylization frame-work built upon a Diffusion Transformer. We first fine-tune a bidirec-tional teacher model on a curated video stylization dataset, supportingboth text-guided and reference-guided video stylization tasks, and sub-sequently distill it into a few-step autoregressive model via post-trainingwith Self Forcing and Distribution Matching Distillation. Furthermore,we propose a reference-preserving KV cache update strategy that notonly enables stable and consistent processing of long videos, but alsosupports real-time switching between text prompts and reference images.Experimental results show that RTR-DiT outperforms existing meth-ods in both text-guided and reference-guided video stylization tasks, interms of quantitative metrics and visual quality, and demonstrates ex-cellent performance in real-time long video stylization and interactivestyle-switching applications.