Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915045022105600 |
|---|---|
| author | Lu, Yaxi Yang, Shenzhi Qian, Cheng Chen, Guirong Luo, Qinyu Wu, Yesai Wang, Huadong Cong, Xin Zhang, Zhong Lin, Yankai Liu, Weiwen Wang, Yasheng Liu, Zhiyuan Liu, Fangming Sun, Maosong |
| author_facet | Lu, Yaxi Yang, Shenzhi Qian, Cheng Chen, Guirong Luo, Qinyu Wu, Yesai Wang, Huadong Cong, Xin Zhang, Zhong Lin, Yankai Liu, Weiwen Wang, Yasheng Liu, Zhiyuan Liu, Fangming Sun, Maosong |
| contents | Agents powered by large language models have shown remarkable abilities in solving complex tasks. However, most agent systems remain reactive, limiting their effectiveness in scenarios requiring foresight and autonomous decision-making. In this paper, we tackle the challenge of developing proactive agents capable of anticipating and initiating tasks without explicit human instructions. We propose a novel data-driven approach for this problem. Firstly, we collect real-world human activities to generate proactive task predictions. These predictions are then labeled by human annotators as either accepted or rejected. The labeled data is used to train a reward model that simulates human judgment and serves as an automatic evaluator of the proactiveness of LLM agents. Building on this, we develop a comprehensive data generation pipeline to create a diverse dataset, ProactiveBench, containing 6,790 events. Finally, we demonstrate that fine-tuning models with the proposed ProactiveBench can significantly elicit the proactiveness of LLM agents. Experimental results show that our fine-tuned model achieves an F1-Score of 66.47% in proactively offering assistance, outperforming all open-source and close-source models. These results highlight the potential of our method in creating more proactive and effective agent systems, paving the way for future advancements in human-agent collaboration. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_12361 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance Lu, Yaxi Yang, Shenzhi Qian, Cheng Chen, Guirong Luo, Qinyu Wu, Yesai Wang, Huadong Cong, Xin Zhang, Zhong Lin, Yankai Liu, Weiwen Wang, Yasheng Liu, Zhiyuan Liu, Fangming Sun, Maosong Artificial Intelligence Computation and Language I.2.7 Agents powered by large language models have shown remarkable abilities in solving complex tasks. However, most agent systems remain reactive, limiting their effectiveness in scenarios requiring foresight and autonomous decision-making. In this paper, we tackle the challenge of developing proactive agents capable of anticipating and initiating tasks without explicit human instructions. We propose a novel data-driven approach for this problem. Firstly, we collect real-world human activities to generate proactive task predictions. These predictions are then labeled by human annotators as either accepted or rejected. The labeled data is used to train a reward model that simulates human judgment and serves as an automatic evaluator of the proactiveness of LLM agents. Building on this, we develop a comprehensive data generation pipeline to create a diverse dataset, ProactiveBench, containing 6,790 events. Finally, we demonstrate that fine-tuning models with the proposed ProactiveBench can significantly elicit the proactiveness of LLM agents. Experimental results show that our fine-tuned model achieves an F1-Score of 66.47% in proactively offering assistance, outperforming all open-source and close-source models. These results highlight the potential of our method in creating more proactive and effective agent systems, paving the way for future advancements in human-agent collaboration. |
| title | Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance |
| topic | Artificial Intelligence Computation and Language I.2.7 |
| url | https://arxiv.org/abs/2410.12361 |