Cover Image

ClickAgent: Enhancing UI Location Capabilities of Autonomous Agents

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hoscilowicz, Jakub, Maj, Bartosz, Kozakiewicz, Bartosz, Tymoshchuk, Oleksii, Janicki, Artur
Format:	Preprint
Published:	2024
Subjects:	Human-Computer Interaction Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

TinyClick: Single-Turn Agent for Empowering GUI Automation
by: Pawlowski, Pawel, et al.
Published: (2024)

ADEPTS: A Capability Framework for Human-Centered Agent Design
by: D'Oro, Pierluca, et al.
Published: (2025)

Known By Their Actions: Fingerprinting LLM Browser Agents via UI Traces
by: Lugoloobi, William, et al.
Published: (2026)

Self-Initiated Open World Learning for Autonomous AI Agents
by: Liu, Bing, et al.
Published: (2021)

Generative UI: LLMs are Effective UI Generators
by: Leviathan, Yaniv, et al.
Published: (2026)

AutoGLM: Autonomous Foundation Agents for GUIs
by: Liu, Xiao, et al.
Published: (2024)

GUIrilla: A Scalable Framework for Automated Desktop UI Exploration
by: Garkot, Sofiya, et al.
Published: (2025)

A Future Capabilities Agent for Tactical Air Traffic Control
by: Kent, Paul, et al.
Published: (2026)

Multi-Agent Reinforcement Learning for Safe Autonomous Driving Under Pedestrian Behavioral Uncertainty
by: Aryan, Prakash, et al.
Published: (2026)

Advancing DRL Agents in Commercial Fighting Games: Training, Integration, and Agent-Human Alignment
by: Zhang, Chen, et al.
Published: (2024)

Agent AI: Surveying the Horizons of Multimodal Interaction
by: Durante, Zane, et al.
Published: (2024)

AI Agents for Inventory Control: Human-LLM-OR Complementarity
by: Baek, Jackie, et al.
Published: (2026)

More Than Irrational: Modeling Belief-Biased Agents
by: Zhu, Yifan, et al.
Published: (2025)

Agent Laboratory: Using LLM Agents as Research Assistants
by: Schmidgall, Samuel, et al.
Published: (2025)

Benchmarking Mobile Device Control Agents across Diverse Configurations
by: Lee, Juyong, et al.
Published: (2024)

A Model for Intelligible Interaction Between Agents That Predict and Explain
by: Baskar, A., et al.
Published: (2023)

Helping Customers in Distress: An LLM-powered Agent that Converses, Probes, and Routes
by: Atreya, Alankar, et al.
Published: (2026)

Personas Evolved: Designing Ethical LLM-Based Conversational Agent Personalities
by: Desai, Smit, et al.
Published: (2025)

Integrating Policy Summaries with Reward Decomposition for Explaining Reinforcement Learning Agents
by: Septon, Yael, et al.
Published: (2022)

UI-JEPA: Towards Active Perception of User Intent through Onscreen User Activity
by: Fu, Yicheng, et al.
Published: (2024)

DAVE: Distribution-aware Attribution via ViT Gradient Decomposition
by: Wróbel, Adam, et al.
Published: (2026)

LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals
by: Park, Joon Sung, et al.
Published: (2024)

SePA: A Search-enhanced Predictive Agent for Personalized Health Coaching
by: Ozolcer, Melik, et al.
Published: (2025)

LLMs May Not Be Human-Level Players, But They Can Be Testers: Measuring Game Difficulty with LLM Agents
by: Xiao, Chang, et al.
Published: (2024)

Classroom Simulacra: Building Contextual Student Generative Agents in Online Education for Learning Behavioral Simulation
by: Xu, Songlin, et al.
Published: (2025)

SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents
by: Cheng, Kanzhi, et al.
Published: (2024)

LLMs as Policy-Agnostic Teammates: A Case Study in Human Proxy Design for Heterogeneous Agent Teams
by: Justus, Aju Ani, et al.
Published: (2025)

Agent-Based Modeling and Deep Neural Networks for Establishing Digital Twins of Secure Facilities under Sensing Restrictions
by: Gunaratne, Chathika, et al.
Published: (2025)

UFO: A UI-Focused Agent for Windows OS Interaction
by: Zhang, Chaoyun, et al.
Published: (2024)

Device-Native Autonomous Agents for Privacy-Preserving Negotiations
by: Roy, Joyjit, et al.
Published: (2026)

Scaling Instructable Agents Across Many Simulated Worlds
by: SIMA Team, et al.
Published: (2024)

Agent-Based Modular Learning for Multimodal Emotion Recognition in Human-Agent Systems
by: Nepomnyaschiy, Matvey, et al.
Published: (2025)

The Odyssey of the Fittest: Can Agents Survive and Still Be Good?
by: Waldner, Dylan, et al.
Published: (2025)

EduAgent: Generative Student Agents in Learning
by: Xu, Songlin, et al.
Published: (2024)

VARP: Reinforcement Learning from Vision-Language Model Feedback with Agent Regularized Preferences
by: Singh, Anukriti, et al.
Published: (2025)

Human-In-the-Loop Software Development Agents
by: Takerngsaksiri, Wannita, et al.
Published: (2024)

Deterministic AI Agent Personality Expression through Standard Psychological Diagnostics
by: Kruijssen, J. M. Diederik, et al.
Published: (2025)

DigiData: Training and Evaluating General-Purpose Mobile Control Agents
by: Sun, Yuxuan, et al.
Published: (2025)

Predicting and Explaining Mobile UI Tappability with Vision Modeling and Saliency Analysis
by: Schoop, Eldon, et al.
Published: (2022)

GhostUI: Unveiling Hidden Interactions in Mobile UI
by: Kweon, Minkyu, et al.
Published: (2026)