RealDyadic: Synthesizing Realistic Dyadic 3D Dialogue with Neural Appearance Priors

Current audio-driven 3D head generation methods primarily focus on single-speaker scenarios, struggling to model natural, bidirectional conversational dynamics. Furthermore, extending these models to dyadic interactions relies heavily on 3D pseudo-labels derived from tracking algorithms. These labels inherently contain severe noise and fail to capture fine-grained facial dynamics, particularly accurate lip closure. To address these limitations, we propose RealDyadic, a two-stage framework that leverages 2D photometric supervision to refine and correct 3D motion priors. In the first stage, a diffusion-based transformer equipped with a dual-audio interaction module generates synchronized 3D facial motions from multi-speaker audio streams. In the second stage, a 3D Gaussian Splatting renderer projects these motions into high-fidelity 2D frames, enabling image-level gradients to backpropagate and penalize the inaccuracies of the initial 3D pseudo-labels via an alternating training strategy. Additionally, we introduce Real-Dialog, a dataset comprising over 50 hours of aligned 2D-3D dyadic conversational data across 500+ identities. Extensive experiments demonstrate that integrating 2D supervision into 3D motion generation significantly outperforms existing baselines in both lip-sync accuracy and conversational realism, explicitly overcoming the performance ceiling dictated by pseudo-label errors. The code is available at https://github.com/Pixel-Talk/RealDyadic.