Skip to yearly menu bar Skip to main content


Poster

Real-time 3D-aware Portrait Editing from a Single Image

Qingyan Bai · Zifan Shi · Yinghao Xu · Hao Ouyang · Qiuyu Wang · Ceyuan Yang · Xuan Wang · Gordon Wetzstein · Yujun Shen · Qifeng Chen

[ ] [ Project Page ]
Wed 2 Oct 1:30 a.m. PDT — 3:30 a.m. PDT

Abstract:

This work presents 3DPE, a practical method that can efficiently edit a face image following given prompts, like reference images or text descriptions, in a 3D-aware manner. To this end, a lightweight module is distilled from a 3D portrait generator and a text-to-image model, which provide prior knowledge of face geometry and superior editing capability, respectively. Such a design brings two compelling advantages over existing approaches. First, our system achieves real-time editing with a feedforward network (i.e., ∼0.04s per image), over 100× faster than the second competitor. Second, thanks to the powerful priors, our module could focus on the learning of editing-related variations, such that it manages to handle various types of editing simultaneously in the training phase and further supports fast adaptation to user-specified customized types of editing during inference (e.g., with ∼5min fine-tuning per style). The code, the model, and the interface will be made publicly available to facilitate future research.

Live content is unavailable. Log in and register to view live content