AgentBay: A Hybrid Interaction Sandbox for Seamless Human-AI Intervention in Agentic Systems
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908693666201600 |
|---|---|
| author | Piao, Yun Min, Hongbo Su, Hang Zhang, Leilei Wang, Lei Yin, Yue Wu, Xiao Xu, Zhejing Qu, Liwei Li, Hang Zeng, Xinxin Tian, Wei Yu, Fei Li, Xiaowei Jiang, Jiayi Liu, Tongxu Tian, Hao Que, Yufei Tu, Xiaobing Suo, Bing Li, Yuebing Chen, Xiangting Zhao, Zeen Tang, Jiaming Huang, Wei Li, Xuguang Zhao, Jing Li, Jin Shen, Jie Ren, Jinkui Zhang, Xiantao |
| author_facet | Piao, Yun Min, Hongbo Su, Hang Zhang, Leilei Wang, Lei Yin, Yue Wu, Xiao Xu, Zhejing Qu, Liwei Li, Hang Zeng, Xinxin Tian, Wei Yu, Fei Li, Xiaowei Jiang, Jiayi Liu, Tongxu Tian, Hao Que, Yufei Tu, Xiaobing Suo, Bing Li, Yuebing Chen, Xiangting Zhao, Zeen Tang, Jiaming Huang, Wei Li, Xuguang Zhao, Jing Li, Jin Shen, Jie Ren, Jinkui Zhang, Xiantao |
| contents | The rapid advancement of Large Language Models (LLMs) is catalyzing a shift towards autonomous AI Agents capable of executing complex, multi-step tasks. However, these agents remain brittle when faced with real-world exceptions, making Human-in-the-Loop (HITL) supervision essential for mission-critical applications. In this paper, we present AgentBay, a novel sandbox service designed from the ground up for hybrid interaction. AgentBay provides secure, isolated execution environments spanning Windows, Linux, Android, Web Browsers, and Code interpreters. Its core contribution is a unified session accessible via a hybrid control interface: An AI agent can interact programmatically via mainstream interfaces (MCP, Open Source SDK), while a human operator can, at any moment, seamlessly take over full manual control. This seamless intervention is enabled by Adaptive Streaming Protocol (ASP). Unlike traditional VNC/RDP, ASP is specifically engineered for this hybrid use case, delivering an ultra-low-latency, smoother user experience that remains resilient even in weak network environments. It achieves this by dynamically blending command-based and video-based streaming, adapting its encoding strategy based on network conditions and the current controller (AI or human). Our evaluation demonstrates strong results in security, performance, and task completion rates. In a benchmark of complex tasks, the AgentBay (Agent + Human) model achieved more than 48% success rate improvement. Furthermore, our ASP protocol reduces bandwidth consumption by up to 50% compared to standard RDP, and in end-to-end latency with around 5% reduction, especially under poor network conditions. We posit that AgentBay provides a foundational primitive for building the next generation of reliable, human-supervised autonomous systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_04367 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AgentBay: A Hybrid Interaction Sandbox for Seamless Human-AI Intervention in Agentic Systems Piao, Yun Min, Hongbo Su, Hang Zhang, Leilei Wang, Lei Yin, Yue Wu, Xiao Xu, Zhejing Qu, Liwei Li, Hang Zeng, Xinxin Tian, Wei Yu, Fei Li, Xiaowei Jiang, Jiayi Liu, Tongxu Tian, Hao Que, Yufei Tu, Xiaobing Suo, Bing Li, Yuebing Chen, Xiangting Zhao, Zeen Tang, Jiaming Huang, Wei Li, Xuguang Zhao, Jing Li, Jin Shen, Jie Ren, Jinkui Zhang, Xiantao Artificial Intelligence The rapid advancement of Large Language Models (LLMs) is catalyzing a shift towards autonomous AI Agents capable of executing complex, multi-step tasks. However, these agents remain brittle when faced with real-world exceptions, making Human-in-the-Loop (HITL) supervision essential for mission-critical applications. In this paper, we present AgentBay, a novel sandbox service designed from the ground up for hybrid interaction. AgentBay provides secure, isolated execution environments spanning Windows, Linux, Android, Web Browsers, and Code interpreters. Its core contribution is a unified session accessible via a hybrid control interface: An AI agent can interact programmatically via mainstream interfaces (MCP, Open Source SDK), while a human operator can, at any moment, seamlessly take over full manual control. This seamless intervention is enabled by Adaptive Streaming Protocol (ASP). Unlike traditional VNC/RDP, ASP is specifically engineered for this hybrid use case, delivering an ultra-low-latency, smoother user experience that remains resilient even in weak network environments. It achieves this by dynamically blending command-based and video-based streaming, adapting its encoding strategy based on network conditions and the current controller (AI or human). Our evaluation demonstrates strong results in security, performance, and task completion rates. In a benchmark of complex tasks, the AgentBay (Agent + Human) model achieved more than 48% success rate improvement. Furthermore, our ASP protocol reduces bandwidth consumption by up to 50% compared to standard RDP, and in end-to-end latency with around 5% reduction, especially under poor network conditions. We posit that AgentBay provides a foundational primitive for building the next generation of reliable, human-supervised autonomous systems. |
| title | AgentBay: A Hybrid Interaction Sandbox for Seamless Human-AI Intervention in Agentic Systems |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2512.04367 |