PhyDetEx: Detecting and Explaining the Physical Plausibility of T2V Models

Driven by the growing capacity and training scale, Text-to-Video (T2V) generation models have recently achieved substantialprogress in video quality, length, and instruction-following capability.However, whether these models can understand physics and generatephysically plausible videos remains a question. While Vision-LanguageModels (VLMs) have been widely used as general-purpose evaluators invarious applications, they struggle to identify the physically impossiblecontent from generated videos. To investigate this issue, we construct aPID (Physical Implausibility Detection) dataset, which consists of a testsplit of 500 manually annotated videos and a train split of 2,588 pairedvideos, where each implausible video is generated by carefully rewritingthe caption of its corresponding real-world video to induce T2V modelsproducing physically implausible content. With the constructed dataset,we introduce a lightweight fine-tuning approach, enabling VLMs to notonly detect physically implausible events but also generate textual expla-nations on the violated physical principles. Taking the fine-tuned VLMas a physical plausibility detector and explainer, namely PhyDetEx,we benchmark a series of state-of-the-art T2V models to assess their ad-herence to physical laws. Our findings show that although recent T2Vmodels have made notable progress toward generating physically plau-sible content, understanding and adhering to physical laws remains achallenging issue, especially for open-source models. Our dataset, train-ing code, and checkpoints are available at https://github.com/Zeqing-Wang/PhyDetEx.