Amplify, Aggregate, and Adjust: VideoMAE-based Holistic-Subtle Aggregation for Micro-Action Recognition

Micro-Action Recognition (MAR) aims to identify transient and subtle bodily movements that occur during interpersonal communication. Existing MAR methods attempt to emphasize low-amplitude motion by injecting skeletal priors and have achieved certain progress. However, they (1) rely on additional modalities and costly pose annotations, and (2) employ coarse joint graphs that fail to capture sub-joint micro-movements and fine-grained appearance cues (e.g., finger tremors). In contrast, raw RGB videos preserve such subtle cues, and pretrained VideoMAE provides strong spatiotemporal priors without requiring additional supervision. Motivated by this, we first establish a strong MAR baseline by task-adaptively fine-tuning VideoMAE. To further enhance its capacity to perceive subtle motion cues, we propose A3-MAE, a VideoMAE-based holistic-subtle collaborative framework that integrates Amplification, Aggregation, and Adjustment in a unified design. Specifically, 1) we design a Temporal Gradient Local Amplification (TGLA) module to amplify subtle-motion regions directly from RGB inputs, forming a subtle-motion branch that complements the holistic-motion counterpart; 2) we develop a Holistic-Subtle Motion Aggregation (HSMA) module with dual cross-attention, enabling reciprocal conditioning between holistic and subtle branches while maintaining their complementarity; and 3) we introduce a Confidence-Aware Dynamic Adjustment (CADA) module to adaptively calibrate aggregation mismatch between the two branches in an instance-aware manner. Together, these components establish a unified amplify-aggregate-adjust paradigm for comprehensive micro-action understanding. Extensive experiments demonstrate that A3-MAE achieves new state-of-the-art performance on both MA-52 and iMiGUE benchmarks. Code is available at https://github.com/zy-hfut/A3MAE.