MARS: Efficient, Adaptive Co-Scheduling for Heterogeneous Agentic Systems
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917448142290944 |
|---|---|
| author | Wang, Yifei Ye, Hancheng Xu, Yechen Guo, Cong Wei, Chiyue Wang, Qinsi Li, Dongting Chen, Tingjun Li, Hai "Helen" Zhuo, Danyang Chen, Yiran |
| author_facet | Wang, Yifei Ye, Hancheng Xu, Yechen Guo, Cong Wei, Chiyue Wang, Qinsi Li, Dongting Chen, Tingjun Li, Hai "Helen" Zhuo, Danyang Chen, Yiran |
| contents | Large language models (LLMs) are increasingly deployed as the execution core of autonomous agents rather than as standalone text generators. Agentic workloads induce a temporal shift from single-turn inference to multi-turn LLM-tool loops, and a spatial shift from chat-scale, GPU-only execution to repository-scale, GPU-CPU co-located execution. Consequently, coordinating heterogeneous resource demands of agentic execution has emerged as a critical system challenge.
We design and implement MARS, an efficient and adaptive co-scheduling system that globally coordinates heterogeneous agentic workloads under coupled GPU-CPU resource pressure. By establishing holistic visibility across GPU inference and CPU tool execution via a unified information stream, an external control plane in MARS decouples admission from execution to prevent heterogeneous resource oversubscription. An internal agent-centric scheduler further minimizes the end-to-end critical path by prioritizing latency-sensitive continuations and adaptively retaining KV cache state only when warm resumption yields a latency benefit. Our evaluations show that MARS reduces end-to-end latency by up to 5.94x while maintaining nearly maximal system throughput. We further integrate MARS as the serving backend for the OpenHands coding agent framework, demonstrating its real-world effectiveness by accelerating end-to-end task completion time by up to 1.87x.
Our source code will be publicly available soon. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_26963 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MARS: Efficient, Adaptive Co-Scheduling for Heterogeneous Agentic Systems Wang, Yifei Ye, Hancheng Xu, Yechen Guo, Cong Wei, Chiyue Wang, Qinsi Li, Dongting Chen, Tingjun Li, Hai "Helen" Zhuo, Danyang Chen, Yiran Operating Systems Distributed, Parallel, and Cluster Computing Machine Learning Multiagent Systems Large language models (LLMs) are increasingly deployed as the execution core of autonomous agents rather than as standalone text generators. Agentic workloads induce a temporal shift from single-turn inference to multi-turn LLM-tool loops, and a spatial shift from chat-scale, GPU-only execution to repository-scale, GPU-CPU co-located execution. Consequently, coordinating heterogeneous resource demands of agentic execution has emerged as a critical system challenge. We design and implement MARS, an efficient and adaptive co-scheduling system that globally coordinates heterogeneous agentic workloads under coupled GPU-CPU resource pressure. By establishing holistic visibility across GPU inference and CPU tool execution via a unified information stream, an external control plane in MARS decouples admission from execution to prevent heterogeneous resource oversubscription. An internal agent-centric scheduler further minimizes the end-to-end critical path by prioritizing latency-sensitive continuations and adaptively retaining KV cache state only when warm resumption yields a latency benefit. Our evaluations show that MARS reduces end-to-end latency by up to 5.94x while maintaining nearly maximal system throughput. We further integrate MARS as the serving backend for the OpenHands coding agent framework, demonstrating its real-world effectiveness by accelerating end-to-end task completion time by up to 1.87x. Our source code will be publicly available soon. |
| title | MARS: Efficient, Adaptive Co-Scheduling for Heterogeneous Agentic Systems |
| topic | Operating Systems Distributed, Parallel, and Cluster Computing Machine Learning Multiagent Systems |
| url | https://arxiv.org/abs/2604.26963 |