Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.13437 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909432327176192 |
|---|---|
| author | Xu, Wenchao Chen, Jinyu Zheng, Peirong Yi, Xiaoquan Tian, Tianyi Zhu, Wenhui Wan, Quan Wang, Haozhao Fan, Yunfeng Su, Qinliang Shen, Xuemin |
| author_facet | Xu, Wenchao Chen, Jinyu Zheng, Peirong Yi, Xiaoquan Tian, Tianyi Zhu, Wenhui Wan, Quan Wang, Haozhao Fan, Yunfeng Su, Qinliang Shen, Xuemin |
| contents | Foundation model (FM) powered agent services are regarded as a promising solution to develop intelligent and personalized applications for advancing toward Artificial General Intelligence (AGI). To achieve high reliability and scalability in deploying these agent services, it is essential to collaboratively optimize computational and communication resources, thereby ensuring effective resource allocation and seamless service delivery. In pursuit of this vision, this paper proposes a unified framework aimed at providing a comprehensive survey on deploying FM-based agent services across heterogeneous devices, with the emphasis on the integration of model and resource optimization to establish a robust infrastructure for these services. Particularly, this paper begins with exploring various low-level optimization strategies during inference and studies approaches that enhance system scalability, such as parallelism techniques and resource scaling methods. The paper then discusses several prominent FMs and investigates research efforts focused on inference acceleration, including techniques such as model compression and token reduction. Moreover, the paper also investigates critical components for constructing agent services and highlights notable intelligent applications. Finally, the paper presents potential research directions for developing real-time agent services with high Quality of Service (QoS). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_13437 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Deploying Foundation Model Powered Agent Services: A Survey Xu, Wenchao Chen, Jinyu Zheng, Peirong Yi, Xiaoquan Tian, Tianyi Zhu, Wenhui Wan, Quan Wang, Haozhao Fan, Yunfeng Su, Qinliang Shen, Xuemin Distributed, Parallel, and Cluster Computing Artificial Intelligence Foundation model (FM) powered agent services are regarded as a promising solution to develop intelligent and personalized applications for advancing toward Artificial General Intelligence (AGI). To achieve high reliability and scalability in deploying these agent services, it is essential to collaboratively optimize computational and communication resources, thereby ensuring effective resource allocation and seamless service delivery. In pursuit of this vision, this paper proposes a unified framework aimed at providing a comprehensive survey on deploying FM-based agent services across heterogeneous devices, with the emphasis on the integration of model and resource optimization to establish a robust infrastructure for these services. Particularly, this paper begins with exploring various low-level optimization strategies during inference and studies approaches that enhance system scalability, such as parallelism techniques and resource scaling methods. The paper then discusses several prominent FMs and investigates research efforts focused on inference acceleration, including techniques such as model compression and token reduction. Moreover, the paper also investigates critical components for constructing agent services and highlights notable intelligent applications. Finally, the paper presents potential research directions for developing real-time agent services with high Quality of Service (QoS). |
| title | Deploying Foundation Model Powered Agent Services: A Survey |
| topic | Distributed, Parallel, and Cluster Computing Artificial Intelligence |
| url | https://arxiv.org/abs/2412.13437 |