OccDirector: Language-Guided Behavior and Interaction Generation in 4D Occupancy Space

Generative world models increasingly rely on 4D occupancyfor realistic autonomous driving simulation. However, existing genera-tion frameworks are limited in two respects: geometric-condition-basedmethods depend on explicit trajectories and cannot generalize to free-form instructions, while text-based methods operate at a coarse attributelevel and fail to orchestrate complex, sequential multi-agent interactions.We propose OccDirector, a framework that generates 4D occupancy dy-namics conditioned solely on natural language scripts, without requir-ing any geometric priors. OccDirector encodes free-form scripts via afrozen VLM and processes occupancy tokens through a Spatio-TemporalMMDiT backbone. A history-prefix anchoring strategy further ensuresthat generated sequences remain consistent with any provided contextover long horizons. To support training and evaluation, we introduceOccInteract-85k, a dataset of 85k clips annotated with multi-level lan-guage instructions spanning static layouts to intricate multi-agent be-haviors, alongside a novel VLM-based evaluation benchmark. Extensiveexperiments demonstrate state-of-the-art generation quality and stronginstruction-following fidelity across diverse driving scenarios.