Parsimonious Flow Matching for Efficient Image Generation

Flow matching (FM) models generate data by learning a ve-locity field that transforms samples from a simple latent distribution,typically an isotropic Gaussian, to the data distribution. However, thegeometric mismatch between a unimodal, full-dimensional Gaussian anda multimodal, approximately low-dimensional data distribution leads toa complex velocity field that is costly to learn at training time andto integrate at inference time. In this paper, we propose ParsimoniousFlow Matching (PFM), which adopts a mixture of Gaussians (MoG)as the latent distribution whose geometry better aligns with the data.We identify key design choices that enable efficient FM, including anoptimal-transport data-latent coupling, MoG estimation via k-means,and eigenvalue regularization of the per-mode covariances. Theoretically,when both data and latent distributions are MoGs, we show that gra-dient descent for FM with affine velocity fields converges faster whenthe corresponding modes of the latent and data are more similar andthe per-mode covariances are well-conditioned. Further, for data thatfollows a separated MoG, we prove that replacing the isotropic Gaus-sian latent with a MoG in FM training accelerates gradient descentconvergence and lowers the initial loss. On CIFAR-10 and ImageNet(32 × 32), PFM achieves the same generation quality as the baselinein fewer training iterations, and produces higher-quality samples for thesame number of function evaluations at inference time. Code is available:https://github.com/tianjiaoding/pfm.