MAVIN: Multi-Shot Audio-Visual Generation with Customized Narrative Control
Abstract
While recent generative models produce high-fidelity videos,they struggle with the complex narrative control required for coherentmulti-shot audio-visual generation. Existing methods suffer from tem-poral misalignment, limited controllability, and incomplete scripting. Inthis paper, we propose MAVIN, the first framework for multi-shot audio-visual generation with customized narrative control. To resolve tempo-ral misalignment, we propose boundary-aware attention, which lever-ages hierarchical captions and boundary-aware token routing to renderaudio-visual elements within their respective temporal boundaries. Toimprove the controllability for multi-subject scenarios, we propose ID-aware propagation, utilizing identity embeddings and an identity-awaremask to bind specific identities to consistent visual appearances and vocaltimbres. To provide comprehensive audio-visual narratives, we present amulti-agent scripting pipeline to transform free-form user inputs into hi-erarchical captions. Furthermore, we construct MAVINSet, a multi-shotaudio-visual dataset for robust training and evaluation. Extensive exper-iments demonstrate that MAVIN achieves state-of-the-art performance,opening up a new avenue for integrating generative models into profes-sional filmmaking workflows.