Cover Image

Large Language Model-Brained GUI Agents: A Survey

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Chaoyun, He, Shilin, Qian, Jiaxu, Li, Bowen, Li, Liqun, Qin, Si, Kang, Yu, Ma, Minghua, Liu, Guyue, Lin, Qingwei, Rajmohan, Saravan, Zhang, Dongmei, Zhang, Qi
Format:	Preprint
Published:	2024
Subjects:	Artificial Intelligence Computation and Language Human-Computer Interaction
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

API Agents vs. GUI Agents: Divergence and Convergence
by: Zhang, Chaoyun, et al.
Published: (2025)

UFO: A UI-Focused Agent for Windows OS Interaction
by: Zhang, Chaoyun, et al.
Published: (2024)

UFO2: The Desktop AgentOS
by: Zhang, Chaoyun, et al.
Published: (2025)

Navigating the Unknown: A Chat-Based Collaborative Interface for Personalized Exploratory Tasks
by: Peng, Yingzhe, et al.
Published: (2024)

Risk-aware Adaptive Virtual CPU Oversubscription in Microsoft Cloud via Prototypical Human-in-the-loop Imitation Learning
by: Wang, Lu, et al.
Published: (2024)

Large Language Models can Deliver Accurate and Interpretable Time Series Anomaly Detection
by: Liu, Jun, et al.
Published: (2024)

GUI Agents: A Survey
by: Nguyen, Dang, et al.
Published: (2024)

LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospects
by: Liu, Guangyi, et al.
Published: (2025)

UFO3: Weaving the Digital Agent Galaxy
by: Zhang, Chaoyun, et al.
Published: (2025)

Characterizing Unintended Consequences in Human-GUI Agent Collaboration for Web Browsing
by: Zhang, Shuning, et al.
Published: (2025)

SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents
by: Cheng, Kanzhi, et al.
Published: (2024)

UIPro: Unleashing Superior Interaction Capability For GUI Agents
by: Li, Hongxin, et al.
Published: (2025)

MagicGUI: A Foundational Mobile GUI Agent with Scalable Data Pipeline and Reinforcement Fine-tuning
by: Tang, Liujian, et al.
Published: (2025)

OmniGUI: Benchmarking GUI Agents in Omni-Modal Smartphone Environments
by: Henry, Felix, et al.
Published: (2026)

GUI Agents with Foundation Models: A Comprehensive Survey
by: Wang, Shuai, et al.
Published: (2024)

A Survey on (M)LLM-Based GUI Agents
by: Tang, Fei, et al.
Published: (2025)

OS-Kairos: Adaptive Interaction for MLLM-Powered GUI Agents
by: Cheng, Pengzhou, et al.
Published: (2025)

MobileViews: A Million-scale and Diverse Mobile GUI Dataset
by: Gao, Longxi, et al.
Published: (2024)

The Behavioral Fabric of LLM-Powered GUI Agents: Human Values and Interaction Outcomes
by: Gebreegziabher, Simret Araya, et al.
Published: (2026)

CRAFT-GUI: Curriculum-Reinforced Agent For GUI Tasks
by: Nong, Songqin, et al.
Published: (2025)

Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight
by: Tang, Jingyu, et al.
Published: (2025)

Toward a Human-Centered Evaluation Framework for Trustworthy LLM-Powered GUI Agents
by: Chen, Chaoran, et al.
Published: (2025)

InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflection
by: Liu, Yuhang, et al.
Published: (2025)

The Obvious Invisible Threat: LLM-Powered GUI Agents' Vulnerability to Fine-Print Injections
by: Chen, Chaoran, et al.
Published: (2025)

AMEX: Android Multi-annotation Expo Dataset for Mobile GUI Agents
by: Chai, Yuxiang, et al.
Published: (2024)

Brain-Muscle Atlas: A novel framework for Motor Brain-Computer Interfaces
by: Sun, Ye, et al.
Published: (2025)

GUI Agents for Continual Game Generation
by: Huang, Yixu, et al.
Published: (2026)

CUPID: Improving Battle Fairness and Position Satisfaction in Online MOBA Games with a Re-matchmaking System
by: Fan, Ge, et al.
Published: (2024)

LearnAct: Few-Shot Mobile GUI Agent with a Unified Demonstration Benchmark
by: Liu, Guangyi, et al.
Published: (2025)

Beyond Chat and Clicks: GUI Agents for In-Situ Assistance via Live Interface Transformation
by: Hao, Pan, et al.
Published: (2026)

GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents
by: Luo, Run, et al.
Published: (2025)

UI-TARS: Pioneering Automated GUI Interaction with Native Agents
by: Qin, Yujia, et al.
Published: (2025)

WinClick: GUI Grounding with Multimodal Large Language Models
by: Hui, Zheng, et al.
Published: (2025)

See, Think, Act: Teaching Multimodal Agents to Effectively Interact with GUI by Identifying Toggles
by: Wu, Zongru, et al.
Published: (2025)

AI-based experts' knowledge visualization of cultural heritage: A case study of Terracotta Warriors
by: Li, Siyi, et al.
Published: (2026)

History-Aware Reasoning for GUI Agents
by: Wang, Ziwei, et al.
Published: (2025)

Affordable Generative Agents
by: Yu, Yangbin, et al.
Published: (2024)

Sensible Agent: A Framework for Unobtrusive Interaction with Proactive AR Agents
by: Lee, Geonsun, et al.
Published: (2025)

ViMo: A Generative Visual GUI World Model for App Agents
by: Luo, Dezhao, et al.
Published: (2025)

TinyClick: Single-Turn Agent for Empowering GUI Automation
by: Pawlowski, Pawel, et al.
Published: (2024)