Semantically Aligned Gradient-Driven Context-Preserving Image Editing

Instruction-guided image editing has a training-time blindspot. Generative editors are never required to semantically verify whethertheir outputs actually satisfy the instruction. Supervision stops at recon-struction and input textual-level conditioning. This produces incompleteedits, spatial spillover, and poor localization. We present IABEdit, amodel-agnostic framework that embeds differentiable semantic verifica-tion into training. A frozen vision-language model extracts spatially-aware descriptors from the ground-truth edit. A trainable aligner thenreproduces them from the generated output. The residual between thetwo becomes a gradient that teaches the generator both what to editand where, with no inference-time VLM cost. IABEdit is compatiblewith diverse backbones, including U-Net (Stable Diffusion) and MMDiT(FLUX), without altering their inference pipelines. On MagicBrush, itimproves structural fidelity by +3.49 DINO-I over the best diffusion base-line and +1.26 over the best overall baseline, while remaining competitiveon instruction alignment. It also achieves state-of-the-art instruction ad-herence performance on RealEdit and EMU Edit benchmarks based onembedding-based metrics. Most consequentially, on the D-LORD surveil-lance benchmark, it surpasses the proprietary Gemini agent by +5.13DINO-P under heavy occlusion, where preserving identity is hardest.This shows that gradient-aligned VLM distillation holds up under real-world-like surveillance and occlusion conditions. Human and GPT-4oevaluations confirm perceptually precise, well-localized edits.