Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2512.08296 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917393689739264 |
|---|---|
| author | Kim, Yubin Gu, Ken Park, Chanwoo Park, Chunjong Schmidgall, Samuel Heydari, A. Ali Yan, Yao Zhang, Zhihan Zhuang, Yuchen Liu, Yun Malhotra, Mark Liang, Paul Pu Park, Hae Won Yang, Yuzhe Xu, Xuhai Du, Yilun Patel, Shwetak Althoff, Tim McDuff, Daniel Liu, Xin |
| author_facet | Kim, Yubin Gu, Ken Park, Chanwoo Park, Chunjong Schmidgall, Samuel Heydari, A. Ali Yan, Yao Zhang, Zhihan Zhuang, Yuchen Liu, Yun Malhotra, Mark Liang, Paul Pu Park, Hae Won Yang, Yuzhe Xu, Xuhai Du, Yilun Patel, Shwetak Althoff, Tim McDuff, Daniel Liu, Xin |
| contents | Agents, language model-based systems capable of reasoning, planning, and acting are widely adopted in real-world tasks, yet how their performance changes as these systems scale across key dimensions remains underexplored. We introduce quantitative scaling principles for agent systems as a predictive model, capturing how performance varies with coordination, model capability, and measurable system and task factors. Across 260 configurations spanning six agentic benchmarks, five canonical architectures (Single-Agent and four Multi-Agent: Independent, Centralized, Decentralized, Hybrid), and three LLM families, we perform controlled evaluations, standardizing tools, prompts, and compute to isolate architectural effects. The resulting model achieves a cross-validated R^2=0.373 across all six benchmarks (R^2=0.413 with a task-grounded capability metric). We identify a robust capability-saturation effect and additional patterns: (1) a coordination yields diminishing returns once single-agent baselines exceed certain performance; (2) tool-heavy tasks appear to incur multi-agent overhead; and (3) architectures without centralized verification tend to propagate errors more than those with centralized coordination. Relative performance change compared to single-agent baseline ranges from +80.8% on decomposable financial reasoning to -70.0% on sequential planning, demonstrating that architecture-task alignment determines collaborative success. The framework identifies the best-performing architecture for 87% of held-out configurations and shows consistent relative architecture preferences on unseen frontier models. Agent effectiveness depends on alignment between coordination and task structure, and that mismatched coordination degrades the performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_08296 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards a Science of Scaling Agent Systems Kim, Yubin Gu, Ken Park, Chanwoo Park, Chunjong Schmidgall, Samuel Heydari, A. Ali Yan, Yao Zhang, Zhihan Zhuang, Yuchen Liu, Yun Malhotra, Mark Liang, Paul Pu Park, Hae Won Yang, Yuzhe Xu, Xuhai Du, Yilun Patel, Shwetak Althoff, Tim McDuff, Daniel Liu, Xin Artificial Intelligence Agents, language model-based systems capable of reasoning, planning, and acting are widely adopted in real-world tasks, yet how their performance changes as these systems scale across key dimensions remains underexplored. We introduce quantitative scaling principles for agent systems as a predictive model, capturing how performance varies with coordination, model capability, and measurable system and task factors. Across 260 configurations spanning six agentic benchmarks, five canonical architectures (Single-Agent and four Multi-Agent: Independent, Centralized, Decentralized, Hybrid), and three LLM families, we perform controlled evaluations, standardizing tools, prompts, and compute to isolate architectural effects. The resulting model achieves a cross-validated R^2=0.373 across all six benchmarks (R^2=0.413 with a task-grounded capability metric). We identify a robust capability-saturation effect and additional patterns: (1) a coordination yields diminishing returns once single-agent baselines exceed certain performance; (2) tool-heavy tasks appear to incur multi-agent overhead; and (3) architectures without centralized verification tend to propagate errors more than those with centralized coordination. Relative performance change compared to single-agent baseline ranges from +80.8% on decomposable financial reasoning to -70.0% on sequential planning, demonstrating that architecture-task alignment determines collaborative success. The framework identifies the best-performing architecture for 87% of held-out configurations and shows consistent relative architecture preferences on unseen frontier models. Agent effectiveness depends on alignment between coordination and task structure, and that mismatched coordination degrades the performance. |
| title | Towards a Science of Scaling Agent Systems |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2512.08296 |