Cover Image

Attention-driven GUI Grounding: Leveraging Pretrained Multimodal Large Language Models without Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xu, Hai-Ming, Chen, Qi, Wang, Lei, Liu, Lingqiao
Format:	Preprint
Published:	2024
Subjects:	Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Parameter-Efficient Fine-Tuning Medical Multimodal Large Language Models for Medical Visual Grounding
by: He, Jinlong, et al.
Published: (2024)

GPT-4 Enhanced Multimodal Grounding for Autonomous Driving: Leveraging Cross-Modal Attention with Large Language Models
by: Liao, Haicheng, et al.
Published: (2023)

Trifuse: Enhancing Attention-Based GUI Grounding via Multimodal Fusion
by: Ma, Longhui, et al.
Published: (2026)

One Last Attention for Your Vision-Language Model
by: Chen, Liang, et al.
Published: (2025)

Exploring Primitive Visual Measurement Understanding and the Role of Output Format in Learning in Vision-Language Models
by: Yadav, Ankit, et al.
Published: (2025)

Towards Fine-Grained Robustness: Attention-Guided Test-Time Prompt Tuning for Vision-Language Models
by: Hai, Jia-Wei, et al.
Published: (2026)

GUI-C$^2$: Coarse-to-Fine GUI Grounding via Difficulty-Aware Reinforcement Learning
by: Li, Junlong, et al.
Published: (2026)

POINTS-GUI-G: GUI-Grounding Journey
by: Zhao, Zhongyin, et al.
Published: (2026)

EndoChat: Grounded Multimodal Large Language Model for Endoscopic Surgery
by: Wang, Guankun, et al.
Published: (2025)

On the Robustness of GUI Grounding Models Against Image Attacks
by: Zhao, Haoren, et al.
Published: (2025)

KARGEN: Knowledge-enhanced Automated Radiology Report Generation Using Large Language Models
by: Li, Yingshu, et al.
Published: (2024)

DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Model
by: Tao, Zhou, et al.
Published: (2025)

Attention-guided Fine-tuning of Multimodal Large Language Models Improves Chain-of-Thought Reasoning
by: Sinha, Sanchit, et al.
Published: (2026)

Towards GUI Agents: Vision-Language Diffusion Models for GUI Grounding
by: Kumbhar, Shrinidhi, et al.
Published: (2026)

Temporal Gains, Spatial Costs: Revisiting Video Fine-Tuning in Multimodal Large Language Models
by: Zhang, Linghao, et al.
Published: (2026)

A Survey on Video Temporal Grounding with Multimodal Large Language Model
by: Wu, Jianlong, et al.
Published: (2025)

FineState-Bench: Benchmarking State-Conditioned Grounding for Fine-grained GUI State Setting
by: Ji, Fengxian, et al.
Published: (2026)

Grounding Everything in Tokens for Multimodal Large Language Models
by: Ren, Xiangxuan, et al.
Published: (2025)

Leveraging Pretrained Diffusion Models for Zero-Shot Part Assembly
by: Zhang, Ruiyuan, et al.
Published: (2025)

Learning Domain Knowledge in Multimodal Large Language Models through Reinforcement Fine-Tuning
by: Cao, Qinglong, et al.
Published: (2026)

Leveraging Large Language Models for Multimodal Search
by: Barbany, Oriol, et al.
Published: (2024)

A Refer-and-Ground Multimodal Large Language Model for Biomedicine
by: Huang, Xiaoshuang, et al.
Published: (2024)

S-RRG-Bench: Structured Radiology Report Generation with Fine-Grained Evaluation Framework
by: Li, Yingshu, et al.
Published: (2025)

Parameter-Efficient Fine-Tuning of Large Pretrained Models for Instance Segmentation Tasks
by: Baker, Nermeen Abou, et al.
Published: (2026)

F-LMM: Grounding Frozen Large Multimodal Models
by: Wu, Size, et al.
Published: (2024)

Language Integration in Fine-Tuning Multimodal Large Language Models for Image-Based Regression
by: Jennings, Roy H., et al.
Published: (2025)

GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding
by: Zhou, Shijie, et al.
Published: (2025)

CogVLM: Visual Expert for Pretrained Language Models
by: Wang, Weihan, et al.
Published: (2023)

OddGridBench: Exposing the Lack of Fine-Grained Visual Discrepancy Sensitivity in Multimodal Large Language Models
by: Weng, Tengjin, et al.
Published: (2026)

ViSurf: Visual Supervised-and-Reinforcement Fine-Tuning for Large Vision-and-Language Models
by: Liu, Yuqi, et al.
Published: (2025)

Confounder-Aware Medical Data Selection for Fine-Tuning Pretrained Vision Models
by: Ji, Anyang, et al.
Published: (2025)

AutoGUI: Scaling GUI Grounding with Automatic Functionality Annotations from LLMs
by: Li, Hongxin, et al.
Published: (2025)

Leveraging Bottom-Up and Top-Down Attention for Few-Shot Object Detection
by: Chen, Xianyu, et al.
Published: (2020)

Hyper Adversarial Tuning for Boosting Adversarial Robustness of Pretrained Large Vision Models
by: Lv, Kangtao, et al.
Published: (2024)

CogAgent: A Visual Language Model for GUI Agents
by: Hong, Wenyi, et al.
Published: (2023)

GUI-Reflection: Empowering Multimodal GUI Models with Self-Reflection Behavior
by: Wu, Penghao, et al.
Published: (2025)

CGL: Advancing Continual GUI Learning via Reinforcement Fine-Tuning
by: Yao, Zhenquan, et al.
Published: (2026)

SinkTrack: Attention Sink based Context Anchoring for Large Language Models
by: Liu, Xu, et al.
Published: (2026)

Q Cache: Visual Attention is Valuable in Less than Half of Decode Layers for Multimodal Large Language Model
by: Zhuang, Jiedong, et al.
Published: (2026)

\textsc{GUI-Spotlight}: Adaptive Iterative Focus Refinement for Enhanced GUI Visual Grounding
by: Lei, Bin, et al.
Published: (2025)