Pythia: Exploiting Workflow Predictability for Efficient Agent-Native LLM Serving
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911682958196736 |
|---|---|
| author | Yu, Shan Shu, Junyi Ni, Yuanjiang Qian, Kun Li, Xue Wang, Yang Zhang, Jinyuan Xu, Ziyi Yang, Shuo Zhu, Lingjun Zhai, Ennan Lu, Qingda Xing, Jiarong Lu, Youyou Jin, Xin Liu, Xuanzhe Xu, Harry |
| author_facet | Yu, Shan Shu, Junyi Ni, Yuanjiang Qian, Kun Li, Xue Wang, Yang Zhang, Jinyuan Xu, Ziyi Yang, Shuo Zhu, Lingjun Zhai, Ennan Lu, Qingda Xing, Jiarong Lu, Youyou Jin, Xin Liu, Xuanzhe Xu, Harry |
| contents | As LLM applications grow more complex, developers are increasingly adopting multi-agent architectures to decompose workflows into specialized, collaborative components, introducing structure that constrains agent behavior and exposes useful semantic predictability. Unlike traditional LLM serving, which operates under highly dynamic and uncertain conditions, this structured topology enables opportunities to reduce runtime uncertainty$\unicode{x2015}$yet existing systems fail to exploit it, treating agentic workloads as generic traffic and incurring significant inefficiencies. Our analysis of production traces from an agent-serving platform and an internal coding assistant reveals key bottlenecks, including low prefix cache hit rates, severe resource contention from long-context requests, and substantial queuing delays due to suboptimal scaling. To address these challenges, we propose Pythia, a multi-agent serving system that captures workflow semantics through a simple interface at the serving layer, unlocking new optimization opportunities and substantially improving throughput and job completion time over state-of-the-art baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_25899 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Pythia: Exploiting Workflow Predictability for Efficient Agent-Native LLM Serving Yu, Shan Shu, Junyi Ni, Yuanjiang Qian, Kun Li, Xue Wang, Yang Zhang, Jinyuan Xu, Ziyi Yang, Shuo Zhu, Lingjun Zhai, Ennan Lu, Qingda Xing, Jiarong Lu, Youyou Jin, Xin Liu, Xuanzhe Xu, Harry Multiagent Systems Distributed, Parallel, and Cluster Computing Systems and Control As LLM applications grow more complex, developers are increasingly adopting multi-agent architectures to decompose workflows into specialized, collaborative components, introducing structure that constrains agent behavior and exposes useful semantic predictability. Unlike traditional LLM serving, which operates under highly dynamic and uncertain conditions, this structured topology enables opportunities to reduce runtime uncertainty$\unicode{x2015}$yet existing systems fail to exploit it, treating agentic workloads as generic traffic and incurring significant inefficiencies. Our analysis of production traces from an agent-serving platform and an internal coding assistant reveals key bottlenecks, including low prefix cache hit rates, severe resource contention from long-context requests, and substantial queuing delays due to suboptimal scaling. To address these challenges, we propose Pythia, a multi-agent serving system that captures workflow semantics through a simple interface at the serving layer, unlocking new optimization opportunities and substantially improving throughput and job completion time over state-of-the-art baselines. |
| title | Pythia: Exploiting Workflow Predictability for Efficient Agent-Native LLM Serving |
| topic | Multiagent Systems Distributed, Parallel, and Cluster Computing Systems and Control |
| url | https://arxiv.org/abs/2604.25899 |