UEval: A Benchmark for Unified Multimodal Generation

We introduce UEval, a benchmark to evaluate unified mod-els, i.e., models capable of generating both images and text. UEval com-prises 1,000 expert-curated questions that require both images and textin the model output, sourced from 8 real-world tasks. Our curated ques-tions cover a wide range of reasoning types, from step-by-step guides totextbook explanations. Evaluating open-ended multimodal generation isnon-trivial, as simple LLM-as-a-judge methods can miss the subtleties.Unlike prior methods relying on multimodal Large Language Models(MLLMs) to rate image quality or text accuracy, we design a rubric-basedscoring system in UEval. For each question, reference images and text an-swers are provided to an MLLM to generate an initial rubric, consisting ofmultiple evaluation criteria, and human experts then refine and validatethese rubrics. In total, UEval contains 10,417 validated rubric criteria,enabling fine-grained automatic scoring. UEval is challenging for currentunified models: GPT-5-Thinking scores only 66.4 out of 100, while thebest open-source model reaches merely 49.1. We observe that reasoningmodels often outperform non-reasoning ones, and transferring reasoningtraces from a reasoning model to a non-reasoning model narrows thegap. This suggests that reasoning may be important for tasks requiringcomplex multimodal understanding and generation. UEval is availableat https://huggingface.co/datasets/zlab-princeton/UEval