Ada-VNNs: Adaptive Equivariance for Vector Neural Networks

Vector Neuron Networks (VNNs) are widely used in 3D tasksfor their data efficiency and strong generalization from equivariance.However, their rigid equivariance constraints hinder handling symmetry-breaking, where low-symmetry outputs must be inferred from highlysymmetric inputs. In this paper, we reveal the representation collapse is-sue in VNNs and propose Ada-VNNs, a vector-neuron architecture thatcan adaptively relax equivariance constraints according to the symmetrylevel of the data. Ada-VNNs introduce an Adaptive Gating Unit (AGU)to control equivariance relaxation in linear layers, and design a resid-ual pathway that transforms hard structural constraints into adjustablesoft priors. We further provide a theoretical characterization that linksthe learned equivariance behavior to intrinsic symmetry breaking in thedata. Experiments on pose estimation across 26 categories with differ-ent degrees of symmetry breaking demonstrate that Ada-VNNs enabledata-driven adaptive equivariance adjustment, alleviate representationcollapse, and significantly improve symmetry-breaking capability overVNN and VN-Transformer, achieving higher performance with almostno additional overhead.