Graph of Agents: Principled Long Context Modeling by Emergent Multi-Agent Collaboration
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911177815097344 |
|---|---|
| author | Joo, Taejong Ishida, Shu Sosnovik, Ivan Lim, Bryan Rezaei-Shoshtari, Sahand Gaier, Adam Giaquinto, Robert |
| author_facet | Joo, Taejong Ishida, Shu Sosnovik, Ivan Lim, Bryan Rezaei-Shoshtari, Sahand Gaier, Adam Giaquinto, Robert |
| contents | As a model-agnostic approach to long context modeling, multi-agent systems can process inputs longer than a large language model's context window without retraining or architectural modifications. However, their performance often heavily relies on hand-crafted multi-agent collaboration strategies and prompt engineering, which limit generalizability. In this work, we introduce a principled framework that formalizes the model-agnostic long context modeling problem as a compression problem, yielding an information-theoretic compression objective. Building on this framework, we propose Graph of Agents (GoA), which dynamically constructs an input-dependent collaboration structure that maximizes this objective. For Llama 3.1 8B and Qwen3 8B across six document question answering benchmarks, GoA improves the average $F_1$ score of retrieval-augmented generation by 5.7\% and a strong multi-agent baseline using a fixed collaboration structure by 16.35\%, respectively. Even with only a 2K context window, GoA surpasses the 128K context window Llama 3.1 8B on LongBench, showing a dramatic increase in effective context length. Our source code is available at https://github.com/tjoo512/graph-of-agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_21848 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Graph of Agents: Principled Long Context Modeling by Emergent Multi-Agent Collaboration Joo, Taejong Ishida, Shu Sosnovik, Ivan Lim, Bryan Rezaei-Shoshtari, Sahand Gaier, Adam Giaquinto, Robert Machine Learning Artificial Intelligence As a model-agnostic approach to long context modeling, multi-agent systems can process inputs longer than a large language model's context window without retraining or architectural modifications. However, their performance often heavily relies on hand-crafted multi-agent collaboration strategies and prompt engineering, which limit generalizability. In this work, we introduce a principled framework that formalizes the model-agnostic long context modeling problem as a compression problem, yielding an information-theoretic compression objective. Building on this framework, we propose Graph of Agents (GoA), which dynamically constructs an input-dependent collaboration structure that maximizes this objective. For Llama 3.1 8B and Qwen3 8B across six document question answering benchmarks, GoA improves the average $F_1$ score of retrieval-augmented generation by 5.7\% and a strong multi-agent baseline using a fixed collaboration structure by 16.35\%, respectively. Even with only a 2K context window, GoA surpasses the 128K context window Llama 3.1 8B on LongBench, showing a dramatic increase in effective context length. Our source code is available at https://github.com/tjoo512/graph-of-agents. |
| title | Graph of Agents: Principled Long Context Modeling by Emergent Multi-Agent Collaboration |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2509.21848 |