MobileManiBench: Simplifying Model Verification for Mobile Manipulation

Vision-language-action models have advanced robotic ma-nipulation but remain constrained by reliance on the large, teleoperation-collected datasets dominated by the static, tabletop scenes. We propose asimulation-first framework to verify VLA architectures before real-worlddeployment and introduce MobileManiBench, a large-scale benchmarkfor mobile-based robotic manipulation. Built on NVIDIA Isaac Sim andpowered by reinforcement learning, our pipeline autonomously gener-ates diverse manipulation trajectories with rich annotations (languageinstructions, multi-view RGB–depth–segmentation images, synchronizedobject/robot states and actions). MobileManiBench features 2 mobileplatforms (parallel-gripper and dexterous-hand robots), 2 synchronizedcameras (head and right wrist), 630 objects in 20 categories, 5 skills(open, close, pull, push, pick) with over 100 tasks performed in 100 re-alistic scenes, yielding 300K trajectories. This design enables controlled,scalable studies of robot embodiments, sensing modalities, and policy ar-chitectures, accelerating research on data efficiency and generalization.We benchmark representative VLA models and report insights into per-ception, reasoning, and control in complex simulated environments, withall code, datasets, and models publicly released at our project website:https://dexhand.github.io/MobileManiBench/.