GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding

Graphical user interface (GUI) grounding is a key capa-bility for computer-use agents, mapping natural-language instructionsto actionable regions on the screen. Existing Multimodal Large Lan-guage Model (MLLM) approaches typically formulate GUI groundingas a text-based coordinate generation task. However, directly gener-ating precise coordinates from visual inputs is challenging and oftendata-intensive. A more intuitive strategy is to first identify instruction-relevant visual patches and then determine the exact click location withinthem. Motivated by recent observations that general MLLMs exhibitnative grounding ability embedded in their attention maps, we pro-pose GUI-AIMA, an attention-based and coordinate-free supervised fine-tuning framework for efficient GUI grounding. GUI-AIMA aligns the in-trinsic multimodal attention of MLLMs with patch-wise grounding sig-nals. These signals are calculated adaptively for diverse user instructionsby multi-head aggregation on simplified query-visual attention matri-ces. Besides, its coordinate-free manner can easily integrate a plug-and-play zoom-in stage. GUI-AIMA-3B was trained with only 509k sam-ples (∼101k screenshots), demonstrating exceptional data efficiency andverifying that light training can trigger the native grounding capabil-ity of MLLMs. It achieves state-of-the-art performance among 3B mod-els, attaining an average accuracy of 61.5% on ScreenSpot-Pro, 92.1%on ScreenSpot-v2, 68.1% on OSWorld-G, 79.1% on MMBench-GUI-L2and 60.0% on UI-Vision. Project page: https://github.com/sjz5202/GUI-AIMA.