FaceArmor: A Universal Facial Image Protection Against Diffusion-Based Manipulations

The rapid advancement of diffusion models has exacerbatedprivacy risks by enabling realistic manipulations of personal facial im-ages. Existing proactive defenses primarily target model-specific internalmodules under restrictive white-box assumptions, rendering their pro-tective efficacy highly vulnerable to unseen architectures and manipula-tion paradigms. To bridge this gap, we propose FaceArmor, a universalfacial image protection framework against both unknown generative ar-chitectures and real-world environmental transformations. Unlike priormodel-dependent approaches, FaceArmor focuses on the intrinsic fea-ture composition of facial images. It leverages an ensemble of surrogateextractors to decompose representations into a structural tier targetinglatent reconstruction and fine textures, and a semantic tier obscuringprompt alignment and biometric identities. These complementary di-mensions are universally exploited by mainstream diffusion-based ma-nipulation methods. FaceArmor then employs an attention-based softweighting scheme to generate effective adversarial perturbations acrossthese heterogeneous features. By extracting intrinsic attention heatmapsdirectly from the surrogate models, this strategy smoothly decouplescompeting gradients and resolves spatial optimization conflicts. Exten-sive experiments demonstrate that FaceArmor not only preserves visualimperceptibility but also achieves exceptional zero-knowledge transfer-ability, robustly thwarting diverse malicious manipulations across state-of-the-art architectures.