Grounding Sim-to-Real Generalization in Dexterous Manipulation: An Empirical Study with Vision-Language-Action Models

Learning a generalist control policy for robotic manipula-tion typically relies on large-scale datasets. Given the high cost of real-world data collection, a practical alternative is to generate synthetic datathrough simulation. However, the resulting synthetic data often exhibitsa significant gap from real-world distributions. While many prior studieshave proposed algorithms to bridge the Sim-to-Real discrepancy, thereremains a lack of principled research that grounds these methods in real-world manipulation tasks, particularly their performance on generalistpolicies such as Vision-Language-Action (VLA) models. In this study,we empirically examine the primary determinants of Sim-to-Real gen-eralization across four dimensions: multi-level domain randomization,photorealistic rendering, physics-realistic modeling, and reinforcementlearning updates. To support this study, we design a comprehensive eval-uation protocol to quantify the real-world performance of manipulationtasks. The protocol accounts for key variations in background, light-ing, distractors, object types, and spatial features. Through experimentsinvolving over 10k real-world trials, we derive critical insights into Sim-to-Real transfer. To inform and advance future studies, we release boththe robotic platforms and the evaluation protocol for public access tofacilitate independent verification, thereby establishing a realistic andstandardized benchmark for robotic manipulation policies.