Interaction as Intelligence Part II: Asynchronous Human-Agent Rollout for Long-Horizon Task Training
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917056306216960 |
|---|---|
| author | Fu, Dayuan Wu, Yunze Cai, Xiaojie Ye, Lyumanshan Xia, Shijie Huang, Zhen Si, Weiye Xu, Tianze Sun, Jie Li, Keyu Jiang, Mohan Wang, Junfei Hua, Qishuo Lu, Pengrui Xiao, Yang Liu, Pengfei |
| author_facet | Fu, Dayuan Wu, Yunze Cai, Xiaojie Ye, Lyumanshan Xia, Shijie Huang, Zhen Si, Weiye Xu, Tianze Sun, Jie Li, Keyu Jiang, Mohan Wang, Junfei Hua, Qishuo Lu, Pengrui Xiao, Yang Liu, Pengfei |
| contents | Large Language Model (LLM) agents have recently shown strong potential in domains such as automated coding, deep research, and graphical user interface manipulation. However, training them to succeed on long-horizon, domain-specialized tasks remains challenging. Current methods primarily fall into two categories. The first relies on dense human annotations through behavior cloning, which is prohibitively expensive for long-horizon tasks that can take days or months. The second depends on outcome-driven sampling, which often collapses due to the rarity of valid positive trajectories on domain-specialized tasks. We introduce Apollo, a sampling framework that integrates asynchronous human guidance with action-level data filtering. Instead of requiring annotators to shadow every step, Apollo allows them to intervene only when the agent drifts from a promising trajectory, by providing prior knowledge, strategic advice, etc. This lightweight design makes it possible to sustain interactions for over 30 hours and produces valuable trajectories at a lower cost. Apollo then applies supervision control to filter out sub-optimal actions and prevent error propagation. Together, these components enable reliable and effective data collection in long-horizon environments. To demonstrate the effectiveness of Apollo, we evaluate it using InnovatorBench. Our experiments show that when applied to train the GLM-4.5 model on InnovatorBench, Apollo achieves more than a 50% improvement over the untrained baseline and a 28% improvement over a variant trained without human interaction. These results highlight the critical role of human-in-the-loop sampling and the robustness of Apollo's design in handling long-horizon, domain-specialized tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_27630 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Interaction as Intelligence Part II: Asynchronous Human-Agent Rollout for Long-Horizon Task Training Fu, Dayuan Wu, Yunze Cai, Xiaojie Ye, Lyumanshan Xia, Shijie Huang, Zhen Si, Weiye Xu, Tianze Sun, Jie Li, Keyu Jiang, Mohan Wang, Junfei Hua, Qishuo Lu, Pengrui Xiao, Yang Liu, Pengfei Artificial Intelligence Large Language Model (LLM) agents have recently shown strong potential in domains such as automated coding, deep research, and graphical user interface manipulation. However, training them to succeed on long-horizon, domain-specialized tasks remains challenging. Current methods primarily fall into two categories. The first relies on dense human annotations through behavior cloning, which is prohibitively expensive for long-horizon tasks that can take days or months. The second depends on outcome-driven sampling, which often collapses due to the rarity of valid positive trajectories on domain-specialized tasks. We introduce Apollo, a sampling framework that integrates asynchronous human guidance with action-level data filtering. Instead of requiring annotators to shadow every step, Apollo allows them to intervene only when the agent drifts from a promising trajectory, by providing prior knowledge, strategic advice, etc. This lightweight design makes it possible to sustain interactions for over 30 hours and produces valuable trajectories at a lower cost. Apollo then applies supervision control to filter out sub-optimal actions and prevent error propagation. Together, these components enable reliable and effective data collection in long-horizon environments. To demonstrate the effectiveness of Apollo, we evaluate it using InnovatorBench. Our experiments show that when applied to train the GLM-4.5 model on InnovatorBench, Apollo achieves more than a 50% improvement over the untrained baseline and a 28% improvement over a variant trained without human interaction. These results highlight the critical role of human-in-the-loop sampling and the robustness of Apollo's design in handling long-horizon, domain-specialized tasks. |
| title | Interaction as Intelligence Part II: Asynchronous Human-Agent Rollout for Long-Horizon Task Training |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2510.27630 |