R2M: Real-Aware Residual Model Merging for Robust and Generalizable Deepfake Detection

Deepfake generators evolve rapidly, making exhaustive datacollection and repeated retraining impractical. Unlike generic multi-tasksettings, deepfake specialists share a common binary objective (Real vs.Fake) and mainly differ in generator-specific artifacts. However, naiveparameter arithmetic can induce unintended decision-boundary shifts,causing unstable ranking behavior and degraded AUC under domainshift. We propose R2 M, a training-free merging framework that decom-poses task vectors into a shared component and generator-specific resid-uals, linking parameter-space updates to logit-space behavior. Offlinespectral construction identifies shared and residual subspaces, and on-line routing selects residuals via first-order gradient-residual alignment,predicting per-sample logit updates under linearization. R2 M is bothefficient and interpretable: expensive computations are performed of-fline, while online inference requires only a single forward–backward passwith lightweight inner-product routing. The same formulation enablesdiagnostic analysis through margin and routing statistics. Experimentsdemonstrate consistently strong performance across in-domain, cross-domain, and unseen settings, highlighting R2 M as an interpretable andscalable approach to training-free deepfake model merging.