Explainable Behavior Cloning: Teaching Large Language Model Agents through Learning by Demonstration
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916461237239808 |
|---|---|
| author | Guan, Yanchu Wang, Dong Wang, Yan Wang, Haiqing Sun, Renen Zhuang, Chenyi Gu, Jinjie Chu, Zhixuan |
| author_facet | Guan, Yanchu Wang, Dong Wang, Yan Wang, Haiqing Sun, Renen Zhuang, Chenyi Gu, Jinjie Chu, Zhixuan |
| contents | Autonomous mobile app interaction has become increasingly important with growing complexity of mobile applications. Developing intelligent agents that can effectively navigate and interact with mobile apps remains a significant challenge. In this paper, we propose an Explainable Behavior Cloning LLM Agent (EBC-LLMAgent), a novel approach that combines large language models (LLMs) with behavior cloning by learning demonstrations to create intelligent and explainable agents for autonomous mobile app interaction. EBC-LLMAgent consists of three core modules: Demonstration Encoding, Code Generation, and UI Mapping, which work synergistically to capture user demonstrations, generate executable codes, and establish accurate correspondence between code and UI elements. We introduce the Behavior Cloning Chain Fusion technique to enhance the generalization capabilities of the agent. Extensive experiments on five popular mobile applications from diverse domains demonstrate the superior performance of EBC-LLMAgent, achieving high success rates in task completion, efficient generalization to unseen scenarios, and the generation of meaningful explanations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_22916 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Explainable Behavior Cloning: Teaching Large Language Model Agents through Learning by Demonstration Guan, Yanchu Wang, Dong Wang, Yan Wang, Haiqing Sun, Renen Zhuang, Chenyi Gu, Jinjie Chu, Zhixuan Computation and Language Autonomous mobile app interaction has become increasingly important with growing complexity of mobile applications. Developing intelligent agents that can effectively navigate and interact with mobile apps remains a significant challenge. In this paper, we propose an Explainable Behavior Cloning LLM Agent (EBC-LLMAgent), a novel approach that combines large language models (LLMs) with behavior cloning by learning demonstrations to create intelligent and explainable agents for autonomous mobile app interaction. EBC-LLMAgent consists of three core modules: Demonstration Encoding, Code Generation, and UI Mapping, which work synergistically to capture user demonstrations, generate executable codes, and establish accurate correspondence between code and UI elements. We introduce the Behavior Cloning Chain Fusion technique to enhance the generalization capabilities of the agent. Extensive experiments on five popular mobile applications from diverse domains demonstrate the superior performance of EBC-LLMAgent, achieving high success rates in task completion, efficient generalization to unseen scenarios, and the generation of meaningful explanations. |
| title | Explainable Behavior Cloning: Teaching Large Language Model Agents through Learning by Demonstration |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.22916 |