Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.09835 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910048120209408 |
|---|---|
| author | Gupta, Naman Singh, Vaibhav Iyer, Arun Shiragur, Kirankumar Grover, Pratham Bairi, Ramakrishna B. Maiti, Ritabrata Damle, Sankarshan Gupta, Shachee Mishra Maurya, Rishikesh C, Vageesh D. |
| author_facet | Gupta, Naman Singh, Vaibhav Iyer, Arun Shiragur, Kirankumar Grover, Pratham Bairi, Ramakrishna B. Maiti, Ritabrata Damle, Sankarshan Gupta, Shachee Mishra Maurya, Rishikesh C, Vageesh D. |
| contents | Sequential multi-agent reasoning frameworks such as Chain-of-Agents (CoA) handle long-context queries by decomposing inputs into chunks and processing them sequentially using LLM-based worker agents that read from and update a bounded shared memory. From a probabilistic perspective, CoA aims to approximate the conditional distribution corresponding to a model capable of jointly reasoning over the entire long context. CoA achieves this through a latent-state factorization in which only bounded summaries of previously processed evidence are passed between agents. The resulting bounded-memory approximation introduces a lossy information bottleneck, making the final evidence state inherently dependent on the order in which chunks are processed.
In this work, we study the problem of chunk ordering for long-context reasoning. We use the well-known Chow-Liu trees to learn a dependency structure that prioritizes strongly related chunks. Empirically, we show that a breadth-first traversal of the resulting tree yields chunk orderings that reduce information loss across agents and consistently outperform both default document-chunk ordering and semantic score-based ordering in answer relevance and exact-match accuracy across three long-context benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_09835 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Chow-Liu Ordering for Long-Context Reasoning in Chain-of-Agents Gupta, Naman Singh, Vaibhav Iyer, Arun Shiragur, Kirankumar Grover, Pratham Bairi, Ramakrishna B. Maiti, Ritabrata Damle, Sankarshan Gupta, Shachee Mishra Maurya, Rishikesh C, Vageesh D. Computation and Language Sequential multi-agent reasoning frameworks such as Chain-of-Agents (CoA) handle long-context queries by decomposing inputs into chunks and processing them sequentially using LLM-based worker agents that read from and update a bounded shared memory. From a probabilistic perspective, CoA aims to approximate the conditional distribution corresponding to a model capable of jointly reasoning over the entire long context. CoA achieves this through a latent-state factorization in which only bounded summaries of previously processed evidence are passed between agents. The resulting bounded-memory approximation introduces a lossy information bottleneck, making the final evidence state inherently dependent on the order in which chunks are processed. In this work, we study the problem of chunk ordering for long-context reasoning. We use the well-known Chow-Liu trees to learn a dependency structure that prioritizes strongly related chunks. Empirically, we show that a breadth-first traversal of the resulting tree yields chunk orderings that reduce information loss across agents and consistently outperform both default document-chunk ordering and semantic score-based ordering in answer relevance and exact-match accuracy across three long-context benchmarks. |
| title | Chow-Liu Ordering for Long-Context Reasoning in Chain-of-Agents |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2603.09835 |