Vulnerability of Privacy-Preserving Visual Localization against Diffusion-based Attacks

The rapid deployment of visual localization (VL) in aug-mented/virtual reality (AR/VR) and autonomous systems makes privacy-preserving localization a critical societal necessity. Existing VL systemsrely on cloud-based 3D scene representation storage and transmit clientside features to a remote server, exposing users to potential reconstruc-tion attacks from intercepted privacy preserving representations. The aimof this paper is therefore to investigate what sensitive information canan adversary actually recover, by comparing multiple privacy-preservingsolutions within the literature. We define privacy as the inability to re-cover personally identifiable information from these representations, ac-knowledging that general scene details do not inherently represent a pri-vacy breach. To model adversarial behavior and to thoroughly measure amethod’s degree of privacy preservation, we introduce a new privacy at-tack that trains a conditional diffusion model to reconstruct images fromprivacy-preserving representations. This reconstruction quality serves asa direct proxy for the amount of sensitive information each represen-tation contains. Leveraging this unified attack protocol, we present thefirst comprehensive comparison across major families of privacy preserv-ing VL methods. Our analysis reveals substantial differences in privacyleakage between representations and highlights limitations in current de-sign assumptions. Code and trained checkpoints may be found here:https://github.com/MaximePi/benchmark-privacy-inversion.