Cover Image

From User Interface to Agent Interface: Efficiency Optimization of UI Representations for LLM Agents

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ran, Dezhi, Gong, Zhi, Guo, Yuzhe, Wu, Mengzhou, Cao, Yuan, Lu, Haochuan, Zhang, Hengyu, Zeng, Xia, Cao, Gang, Yao, Liangchao, Deng, Yuetang, Yang, Wei, Xie, Tao
Format:	Preprint
Published:	2025
Subjects:	Software Engineering Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Beyond Pass or Fail: Multi-Dimensional Benchmarking of Foundation Models for Goal-based Mobile UI Navigation
by: Ran, Dezhi, et al.
Published: (2025)

UI-Oceanus: Scaling GUI Agents with Synthetic Environmental Dynamics
by: Wu, Mengzhou, et al.
Published: (2026)

Skill-Adpative Imitation Learning for UI Test Reuse
by: Wu, Mengzhou, et al.
Published: (2024)

An Infrastructure Software Perspective Toward Computation Offloading between Executable Specifications and Foundation Models
by: Ran, Dezhi, et al.
Published: (2025)

Enabling Cost-Effective UI Automation Testing with Retrieval-Based LLMs: A Case Study in WeChat
by: Feng, Sidong, et al.
Published: (2024)

UI Semantic Group Detection: Grouping UI Elements with Similar Semantics in Mobile Graphical User Interface
by: Xiao, Shuhong, et al.
Published: (2024)

Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software
by: Ran, Dezhi, et al.
Published: (2024)

Hidden Technical Debt in Generative (GenUI) and Malleable User Interfaces
by: Cifliku, Besjon
Published: (2026)

AppForge: From Assistant to Independent Developer -- Are GPTs Ready for Software Development?
by: Ran, Dezhi, et al.
Published: (2025)

EmbeWebAgent: Embedding Web Agents into Any Customized UI
by: Ma, Chenyang, et al.
Published: (2026)

GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training
by: Cao, Yuan, et al.
Published: (2026)

Zero-Shot Prompting Approaches for LLM-based Graphical User Interface Generation
by: Kolthoff, Kristian, et al.
Published: (2024)

From Human Interfaces to Agent Interfaces: Rethinking Software Design in the Age of AI-Native Systems
by: Wang, Shaolin, et al.
Published: (2026)

Towards LLM-Based Usability Analysis for Recommender User Interfaces
by: Lubos, Sebastian, et al.
Published: (2025)

Agent for User: Testing Multi-User Interactive Features in TikTok
by: Feng, Sidong, et al.
Published: (2025)

ASA: Training-Free Representation Engineering for Tool-Calling Agents
by: Wang, Youjin, et al.
Published: (2026)

SkillReducer: Optimizing LLM Agent Skills for Token Efficiency
by: Gao, Yudong, et al.
Published: (2026)

An Autonomous RL Agent Methodology for Dynamic Web UI Testing in a BDD Framework
by: Mughal, Ali Hassaan
Published: (2025)

SeaView: Software Engineering Agent Visual Interface for Enhanced Workflow
by: Bula, Timothy, et al.
Published: (2025)

SkillSmith: Compiling Agent Skills into Boundary-Guided Runtime Interfaces
by: Xu, Duling, et al.
Published: (2026)

Post Processing Graphical User Interface for Heat Flow Visualization
by: Olt, Lars, et al.
Published: (2025)

GraphCodeAgent: Dual Graph-Guided LLM Agent for Retrieval-Augmented Repo-Level Code Generation
by: Li, Jia, et al.
Published: (2025)

JSidentify-V2: Leveraging Dynamic Memory Fingerprinting for Mini-Game Plagiarism Detection
by: Li, Zhihao, et al.
Published: (2025)

Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
by: Merrill, Mike A., et al.
Published: (2026)

Efficiency, Expressivity, and Extensibility in a Close-to-Metal NPU Programming Interface
by: Hunhoff, Erika, et al.
Published: (2025)

Automatically Benchmarking LLM Code Agents through Agent-Driven Annotation and Evaluation
by: Fu, Lingyue, et al.
Published: (2025)

From LLMs to Agents: A Comparative Evaluation of LLMs and LLM-based Agents in Security Patch Detection
by: Han, Junxiao, et al.
Published: (2025)

Managing Uncertainty in LLM-based Multi-Agent System Operation
by: Zhang, Man, et al.
Published: (2026)

AutoMT: A Multi-Agent LLM Framework for Automated Metamorphic Testing of Autonomous Driving Systems
by: Liang, Linfeng, et al.
Published: (2025)

SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
by: Yang, John, et al.
Published: (2024)

ReuseDroid: A VLM-empowered Android UI Test Migrator Boosted by Active Feedback
by: Li, Xiaolei, et al.
Published: (2025)

Agentless: Demystifying LLM-based Software Engineering Agents
by: Xia, Chunqiu Steven, et al.
Published: (2024)

Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agents
by: Chen, Zhi, et al.
Published: (2026)

UITrans: Seamless UI Translation from Android to HarmonyOS
by: Gong, Lina, et al.
Published: (2024)

JSProtect: A Scalable Obfuscation Framework for Mini-Games in WeChat
by: Li, Zhihao, et al.
Published: (2025)

Reactive Semantics for User Interface Description Languages
by: Pesin, Basile, et al.
Published: (2025)

AgentDroid: A Multi-Agent Framework for Detecting Fraudulent Android Applications
by: Pan, Ruwei, et al.
Published: (2025)

A Deep Dive into Retrieval-Augmented Generation for Code Completion: Experience on WeChat
by: Yang, Zezhou, et al.
Published: (2025)

Common Sense and User Interfaces: Issues beyond the Keyboard.
by: Crawford, Walt
Published: (1988)

Benchmarking and Studying the LLM-based Agent System in End-to-End Software Development
by: Zeng, Zhengran, et al.
Published: (2025)