Trust Your Instincts: Confidence-Driven Test-Time RL for Vision-Language-Action Models

Reinforcement learning (RL) has become indispensable forpushing Vision-Language-Action Models (VLA) beyond static imitationlearning. However, existing RL methods typically necessitate externalenvironmental feedback, relying on predefined success signals to guidepolicy updates. In this work, we demonstrate that VLA models possessstrong internal evaluative capabilities: in discrete-action VLAs, trajecto-ries with higher generation confidence are significantly more likely to suc-ceed. Based on the observation, we introduce T2 VLA (Test-time VLA),an architecture-agnostic test-time RL framework that enables VLA mod-els to achieve self-bootstrapping policy improvement. Instead of relyingon external rewards, T2 VLA leverages the trajectory-level similarityto high-confidence expert demonstrations as an intrinsic reward signal.In addition, we propose a Confidence-Driven Dual Expert Bootstrappingmechanism. By dynamically balancing a Local Pseudo-Expert for aggres-sive exploration and a Global Expert Pool for training stability, T2 VLAprevents policy collapse while discovering further breakthroughs. Exten-sive experiments on the LIBERO and RoboTwin benchmarks show thatT2 VLA consistently outperforms supervised baselines and approachesoracle RL performance with ground-truth rewards, achieving effectiveimprovement without external reward feedback. Furthermore, T2 VLAcan adapt to distinct VLA paradigms, including both OpenVLA-OFTand the π series.