NearID: Identity Representation Learning via Near-identity Distractors

When evaluating identity-focused tasks such as personalizedgeneration and image editing, existing vision encoders entangle objectidentity with background context, leading to unreliable representationsand metrics. We introduce a principled framework for matched-contextidentity confusion using Near-identity (NearID) distractors, where se-mantically similar but distinct instances are placed on the exact samebackground as a reference image, eliminating contextual shortcuts andisolating identity as the sole discriminative signal. Based on this prin-ciple, we present the NearID dataset (19K identities, 316K matched-context distractors) together with a strict margin-based evaluation pro-tocol. Under this setting, pre-trained encoders perform poorly, achievingSample Success Rates (SSR), a strict margin-based identity discrimina-tion metric, as low as 30.7% and often ranking distractors above truecross-view matches. We address this by learning identity-aware repre-sentations on a frozen backbone using a two-tier contrastive objectiveenforcing the hierarchy: same identity > NearID distractor > randomnegative. This improves SSR to 99.2%, enhances part-level discrimina-tion by 28.0%, and yields stronger alignment with human judgments onDreamBench++, a human-aligned benchmark for personalization.