Beyond Atomic Layouts: Compositional Design Understanding with Vision-Language Models

Layout understanding, or the interpretation of element orga-nization, is essential for document analysis, user interface (UI) creation,and graphic design. While recent vision-language models (VLMs) ex-cel at interpreting atomic layouts composed of independent elements,they struggle with compositional layouts that require reasoning over vi-sually entangled elements within hierarchical multi-layer structures. Inthis paper, we introduce a new task, compositional layout understanding,and present CoDeLayout, a VQA dataset of ∼20K real-world multi-layerlayouts annotated with compositional element pairs and design intent.Through empirical analysis on CoDeLayout, we identify two key chal-lenges for existing VLMs: semantic drift between textual metadata andvisual content, and structural ambiguity in hierarchical inter-elementrelationships. To address these challenges, we propose MASON, a post-training paradigm that integrates multimodal alignment (MA) and struc-tural perception (SP). MA enhances element interpretation by ground-ing metadata-defined elements to their visual counterparts, mitigatingsemantic drift, while SP models layer-aware inter-element spatial rela-tionships to improve hierarchical understanding and reduce structuralambiguity. Experiments reveal substantial gaps in existing VLMs: eventhe strongest baseline, GPT-o3, achieves only 79.68% accuracy, whereasQwen2.5-VL 7B with MASON reaches 91.66%. Notably, MASON sur-passes full-data Direct Finetune using only 30% of the training data andscales better with additional data.