Semantic Browsing: Controllable Diversity for Image Generation

Modern text-to-image models excel in visual fidelity andprompt adherence. However, this strict adherence comes at the cost ofdiversity: generated samples tend to collapse into a single visual inter-pretation. Existing methods to improve diversity produce outputs drivenby incidental variations rather than meaningful design choices. This mo-tivates a new variant of the diversity task where structure is enforced onthe generated samples.We introduce a method for controlled diversity that enables SemanticBrowsing, where users can navigate structured image galleries and expe-rience creative exploration through a systematic traversal of meaningful,interpretable axes of variation. Achieving this level of semantic controlrequires a deep understanding of the scene. We exploit the fact thatrecent text-to-image models are trained on elaborated captions, effec-tively decoupling semantic decision-making from pixel generation. Thisenables a paradigm shift: instead of relying on stochastic variation withinthe text-to-image model, we induce diversity directly at the text level.By leveraging rich textual representations, we allow a Vision LanguageModel (VLM) to operate on the full scene context. To overcome thegeneric outputs typical of standard VLMs, we employ an agentic work-flow that explicitly enforces structured variation attuned to the originalprompt. We demonstrate that our method produces diverse and naviga-ble design spaces where every variation corresponds to a specific, user-understandable semantic decision.