Imagen de Portada

Beyond Chat and Clicks: GUI Agents for In-Situ Assistance via Live Interface Transformation

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Hao, Pan, Selvakumaran, Rishi, Sun, Jacob, Wang, Qianwen
Formato:	Preprint
Publicado:	2026
Materias:	Human-Computer Interaction
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents
por: Cheng, Kanzhi, et al.
Publicado: (2024)

Beyond Clicking:A Step Towards Generalist GUI Grounding via Text Dragging
por: Liao, Zeyi, et al.
Publicado: (2025)

TinyClick: Single-Turn Agent for Empowering GUI Automation
por: Pawlowski, Pawel, et al.
Publicado: (2024)

WinClick: GUI Grounding with Multimodal Large Language Models
por: Hui, Zheng, et al.
Publicado: (2025)

The Pen: Episodic Cognitive Assistance via an Ear-Worn Interface
por: Tussa, Yonatan, et al.
Publicado: (2026)

FlowForge: Guiding the Creation of Multi-agent Workflows with Design Space Visualization as a Thinking Scaffold
por: Hao, Pan, et al.
Publicado: (2025)

Probing the Gaps in ChatGPT Live Video Chat for Real-World Assistance for People who are Blind or Visually Impaired
por: Chang, Ruei-Che, et al.
Publicado: (2025)

LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospects
por: Liu, Guangyi, et al.
Publicado: (2025)

Not All Uncertainty Is Equal: How Uncertainty Granularity Shapes Human Verification in LLM-Assisted Decision Making
por: Villavicencio, Mauricio, et al.
Publicado: (2026)

A User-Centered Teleoperation GUI for Automated Vehicles: Identifying and Evaluating Information Requirements for Remote Driving and Assistance
por: Wolf, Maria-Magdalena, et al.
Publicado: (2025)

Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight
por: Tang, Jingyu, et al.
Publicado: (2025)

LearnAct: Few-Shot Mobile GUI Agent with a Unified Demonstration Benchmark
por: Liu, Guangyi, et al.
Publicado: (2025)

GUI Agents with Foundation Models: A Comprehensive Survey
por: Wang, Shuai, et al.
Publicado: (2024)

CRAFT-GUI: Curriculum-Reinforced Agent For GUI Tasks
por: Nong, Songqin, et al.
Publicado: (2025)

GUI Agents: A Survey
por: Nguyen, Dang, et al.
Publicado: (2024)

OmniGUI: Benchmarking GUI Agents in Omni-Modal Smartphone Environments
por: Henry, Felix, et al.
Publicado: (2026)

AgentClick: A Skill-Based Human-in-the-Loop Review Layer for Terminal AI Agents
por: Zhuang, Haomin, et al.
Publicado: (2026)

The Illusion of Agreement with ChatGPT: Sycophancy and Beyond
por: Noshin, Kazi, et al.
Publicado: (2026)

Characterizing Unintended Consequences in Human-GUI Agent Collaboration for Web Browsing
por: Zhang, Shuning, et al.
Publicado: (2025)

ProAgentBench: Evaluating LLM Agents for Proactive Assistance with Real-World Data
por: Tang, Yuanbo, et al.
Publicado: (2026)

In-Browser Agents for Search Assistance
por: Zerhoudi, Saber, et al.
Publicado: (2026)

AmbiBench: Benchmarking Mobile GUI Agents Beyond One-Shot Instructions in the Wild
por: Sun, Jiazheng, et al.
Publicado: (2026)

SpiritSight Agent: Advanced GUI Agent with One Look
por: Huang, Zhiyuan, et al.
Publicado: (2025)

The Behavioral Fabric of LLM-Powered GUI Agents: Human Values and Interaction Outcomes
por: Gebreegziabher, Simret Araya, et al.
Publicado: (2026)

Beyond Cybathlon: On-demand Quadrupedal Assistance for People with Limited Mobility
por: Scheidemann, Carmen, et al.
Publicado: (2026)

MagicGUI: A Foundational Mobile GUI Agent with Scalable Data Pipeline and Reinforcement Fine-tuning
por: Tang, Liujian, et al.
Publicado: (2025)

API Agents vs. GUI Agents: Divergence and Convergence
por: Zhang, Chaoyun, et al.
Publicado: (2025)

From Latent to Observable Position-Based Click Models in Carousel Interfaces
por: de Leon-Martinez, Santiago, et al.
Publicado: (2026)

Beyond Benchmarks: How Users Evaluate AI Chat Assistants
por: Awan, Moiz Sadiq, et al.
Publicado: (2026)

KNOWNET: Guided Health Information Seeking from LLMs via Knowledge Graph Integration
por: Yan, Youfu, et al.
Publicado: (2024)

A Design Space for Live Music Agents
por: Kim, Yewon, et al.
Publicado: (2026)

Beyond the Click: A Framework for Inferring Cognitive Traces in Search
por: Zerhoudi, Saber, et al.
Publicado: (2026)

LADICA: A Large Shared Display Interface for Generative AI Cognitive Assistance in Co-Located Team Collaboration
por: Zhang, Zheng, et al.
Publicado: (2024)

More AI Assistance Reduces Cognitive Engagement: Examining the AI Assistance Dilemma in AI-Supported Note-Taking
por: Chen, Xinyue, et al.
Publicado: (2025)

PANDA: Parkinson's Assistance and Notification Driving Aid
por: Wen, Tianyang, et al.
Publicado: (2025)

WeldAR: Augmenting Live Hands-On Training with In-Situ Guidance for Novice Learners
por: Xu, Chuhan, et al.
Publicado: (2026)

Learning to Live with AI: How Students Develop AI Literacy Through Naturalistic ChatGPT Interaction
por: Ammari, Tawfiq, et al.
Publicado: (2026)

Learning Agent-based Modeling with LLM Companions: Experiences of Novices and Experts Using ChatGPT & NetLogo Chat
por: Chen, John, et al.
Publicado: (2024)

Speculating About Multi-user Conversational Interfaces and LLMs: What If Chatting Wasn't So Lonely?
por: Seymour, William, et al.
Publicado: (2024)

Living Contracts: Beyond Document-Centric Interaction with Legal Agreements
por: Huang, Ziheng, et al.
Publicado: (2026)