Tri-Efficient Transfer Learning for Point Cloud Videos
Abstract
While point cloud foundation models have significantly ad-vanced point cloud video understanding, existing parameter-efficient fine-tuning (PEFT) methods still suffer from two critical limitations: pro-hibitive annotation costs for large-scale point cloud datasets and severememory bottlenecks. In this paper, we aim to mine richer supervisionsignals from existing data rather than blindly scaling datasets. A furtherkey principle is that the memory footprint of fine-tuning must be dras-tically reduced compared to full fine-tuning, which remains elusive forcurrent PEFT techniques. Driven by these challenges, we identify threecore desiderata: data-, parameter-, and memory efficiency, and presentPoinTriE, a unified framework that excels along all three dimensions.For pre-training, pseudo-motion trajectories are synthesized via rigidtransformations, paired with text corpora and 2D projections derivedfrom raw point clouds. We then propose a Geometric-Motion DualityNetwork optimized via multimodal contrastive learning, rigid rotationprediction, and motion distribution divergence to produce dense self-supervision. During fine-tuning, we freeze the pretrained backbone andonly update a lightweight Spatio-temporal Side Network built with LoRAunits. Equipped with a gradient flow masking strategy, PoinTriE simul-taneously reduces memory consumption and parameter overhead. Exten-sive experiments confirm that PoinTriE establishes new state-of-the-artresults on action recognition and semantic segmentation tasks.