Exploring Guided Sampling of Conditional GANs

Guided sampling serves as a widely used inference technique in diffusion models to trade off the sample fidelity and diversity.In this work, we confirm that generative adversarial networks (GANs) can also benefit from guided sampling, not even requiring to pre-prepare a classifier (\textit{i.e.}, classifier guidance) or learn an unconditional counterpart (\textit{i.e.}, classifier-free guidance) as in diffusion models. Inspired by the organized latent space in GANs, we manage to estimate the data-condition joint distribution from a well-learned conditional generator simply through vector arithmetic. With such an \textit{easy implementation}, our approach, termed \method, improves the FID score of a state-of-the-art GAN model pre-trained on ImageNet $64\times64$ from 8.87 to 6.06, barely increasing the inference time. We then propose a learning-based variant of our framework to better approximate the distribution of the entire dataset, further improving the FID score to 4.37. It is noteworthy that our sampling strategy sufficiently closes the gap between GANs and one-step diffusion models (\textit{i.e.}, with FID 4.02) under comparable model size. We will release the code to facilitate future studies.