AIOS: LLM Agent Operating System
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915441876664320 |
|---|---|
| author | Mei, Kai Zhu, Xi Xu, Wujiang Hua, Wenyue Jin, Mingyu Li, Zelong Xu, Shuyuan Ye, Ruosong Ge, Yingqiang Zhang, Yongfeng |
| author_facet | Mei, Kai Zhu, Xi Xu, Wujiang Hua, Wenyue Jin, Mingyu Li, Zelong Xu, Shuyuan Ye, Ruosong Ge, Yingqiang Zhang, Yongfeng |
| contents | LLM-based intelligent agents face significant deployment challenges, particularly related to resource management. Allowing unrestricted access to LLM or tool resources can lead to inefficient or even potentially harmful resource allocation and utilization for agents. Furthermore, the absence of proper scheduling and resource management mechanisms in current agent designs hinders concurrent processing and limits overall system efficiency. To address these challenges, this paper proposes the architecture of AIOS (LLM-based AI Agent Operating System) under the context of managing LLM-based agents. It introduces a novel architecture for serving LLM-based agents by isolating resources and LLM-specific services from agent applications into an AIOS kernel. This AIOS kernel provides fundamental services (e.g., scheduling, context management, memory management, storage management, access control) for runtime agents. To enhance usability, AIOS also includes an AIOS SDK, a comprehensive suite of APIs designed for utilizing functionalities provided by the AIOS kernel. Experimental results demonstrate that using AIOS can achieve up to 2.1x faster execution for serving agents built by various agent frameworks. The source code is available at https://github.com/agiresearch/AIOS. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_16971 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AIOS: LLM Agent Operating System Mei, Kai Zhu, Xi Xu, Wujiang Hua, Wenyue Jin, Mingyu Li, Zelong Xu, Shuyuan Ye, Ruosong Ge, Yingqiang Zhang, Yongfeng Operating Systems Artificial Intelligence Computation and Language LLM-based intelligent agents face significant deployment challenges, particularly related to resource management. Allowing unrestricted access to LLM or tool resources can lead to inefficient or even potentially harmful resource allocation and utilization for agents. Furthermore, the absence of proper scheduling and resource management mechanisms in current agent designs hinders concurrent processing and limits overall system efficiency. To address these challenges, this paper proposes the architecture of AIOS (LLM-based AI Agent Operating System) under the context of managing LLM-based agents. It introduces a novel architecture for serving LLM-based agents by isolating resources and LLM-specific services from agent applications into an AIOS kernel. This AIOS kernel provides fundamental services (e.g., scheduling, context management, memory management, storage management, access control) for runtime agents. To enhance usability, AIOS also includes an AIOS SDK, a comprehensive suite of APIs designed for utilizing functionalities provided by the AIOS kernel. Experimental results demonstrate that using AIOS can achieve up to 2.1x faster execution for serving agents built by various agent frameworks. The source code is available at https://github.com/agiresearch/AIOS. |
| title | AIOS: LLM Agent Operating System |
| topic | Operating Systems Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2403.16971 |