CaRe: Critical Parameter Rectification for Efficient Visual Modeling

Designing efficient vision backbones is increasingly challeng-ing. An intriguing avenue is mergeable re-parameterization, which canreshape training dynamics by introducing auxiliary parameterizationswithout inference overhead. In particular, RepVGG-style structures can beviewed as adding a fixed-support correction on top of a dense base kernelafter merging. In this paper, we propose Critical Parameter Rectification(CaRe), which generalizes fixed corrections to learned selective rectifi-cation. CaRe factorizes each layer as W = Wbase + Waux ⊙ ϕ(Wgate ),where a static parameter-space gate induces sparse activation in thecorrection term, concentrating updates on a small subset of critical pa-rameters during training. Since the gate depends only on parameters,CaRe is losslessly mergeable into a standard weight tensor, incurring zeroinference overhead. Built on CaRe, the CaReNet family establishes astrong accuracy-throughput Pareto frontier on ImageNet-1K, with consis-tent gains across model scales and strong transfer to downstream denseprediction tasks. Code is available at https://github.com/lime-j/care.