Taming LLMs for Codematic Indoor Scene Generation
Abstract
The generation of plausible layouts is a critical step in textdriven 3D scene synthesis. While Large Language Models (LLMs) have shown promise in high-level organization in this task, their native finegrained spatial placement remains a significant bottleneck, often requiring in-context guidance or complex agentic reflections to fix errors. This paper focuses directly on strengthening the core spatial ability by finetuning LLM to generate structured scene code (codematic indoor scene generation). We identify and address two fundamental issues hindering the LLM from doing this: data scarcity and poor instruction following. First, to mitigate data scarcity, we consolidate multiple heterogeneous datasets, spanning procedural, real-world, and professionally designed scenes, into a unified, large-scale training corpus. This dataset comprises 52K rooms and 280k data pairs, featuring diverse descriptions at both the scene and object levels. Second, even with abundant data, LLMs exhibit poor instruction following due to an information imbalance, where the tokenheavy layout history overwhelms the concise user prompt. To resolve this, we propose SceneSpinner, a framework that introduces a language-based planning stage to provide high-level reasoning and a novel Conditional Mutual Information (CMI) regularization to explicitly force the model to focus on user instructions. Experiments demonstrate that our approach significantly improves the ability of LLMs to generate plausible, diverse, and instruction-aligned 3D layouts.