SceneOrchestra: Efficient Agentic 3D Scene Synthesis via Full Tool-Call Trajectory Generation

Recent agentic frameworks for 3D scene synthesis have ad-vanced realism and diversity by integrating heterogeneous generationand editing tools. These tools are organized into workflows orchestratedby an off-the-shelf LLM. Current approaches typically adopt an exe-cute–review–reflect loop: at each step, the orchestrator executes a tool,renders intermediate results for review, and then decides on the tooland its parameters for the next step. However, this design has two keylimitations. First, next-step tool selection and parameter configurationare driven by heuristic rules, which can lead to suboptimal executionflows, unnecessary tool invocations, degraded output quality, and in-creased runtime. Second, rendering and reviewing intermediate resultsafter each step introduces additional latency. To address these issues, wepropose SceneOrchestra, a trainable orchestration framework that opti-mizes the tool-call execution flow and eliminates the step-by-step reviewloop, improving both efficiency and output quality. SceneOrchestra con-sists of an orchestrator and a discriminator, which we fine-tune witha two-phase training strategy. In the first phase, the orchestrator learnscontext-aware tool selection and complete tool-call trajectory generation,while the discriminator is trained to assess the quality of full trajectories,enabling it to select the best trajectory from multiple candidates. In thesecond phase, we perform interleaved training, where the discriminatoradapts to the orchestrator’s evolving trajectory distribution and distillsits discriminative capability back into the orchestrator. At inference, weonly use the orchestrator to generate and execute full tool-call trajec-tories from instructions, without requiring the discriminator. Extensiveexperiments show that our method achieves state-of-the-art scene qualitycompared to previous work.