Diffusion-Based Image-to-Image Translation by Noise Correction via Prompt Interpolation

We propose a simple but effective training-free method tailored to diffusion-based image-to-image translation. Our approach revises the original noise prediction network of a diffusion model by incorporating a noise correction term, which is based on the progressive interpolation of the textual prompts corresponding to a given source image and a desired target image. We formulate the noise correction term as the difference between two noise predictions; one is computed from the denoising network given a target latent and an adaptive interpolation between the source and target prompt embeddings, while the other is the noise prediction given the target latent and the source prompt embedding. The final noise prediction network is given by a combination of the standard denoising term and the noise correction term, where the firmer is designed to reconstruct must-be-preserved regions while the latter aims to effectively edit regions of interest relevant to the target prompt. Extensive experiments verify that the proposed method achieves outstanding performance with fast inference time and consistently improves existing frameworks when combined with them.