When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911474904989696 |
|---|---|
| author | Xu, Zhen Zhu, Shang Wang, Jue Wang, Junlin Athiwaratkun, Ben Wang, Chi Zou, James Zhang, Ce |
| author_facet | Xu, Zhen Zhu, Shang Wang, Jue Wang, Junlin Athiwaratkun, Ben Wang, Chi Zou, James Zhang, Ce |
| contents | We investigate the challenge of applying Large Language Models (LLMs) to long texts. We propose a theoretical framework that distinguishes the failure modes of long context tasks into three categories: cross-chunk dependence (task noise), confusion that grows with context size (model noise), and the imperfect integration of partial results (aggregator noise). Under this view, we analyze when it is effective to use multi-agent chunking, i.e., dividing a lengthy sequence into smaller chunks and aggregating the processed results of each chunk. Our experiments on tasks such as retrieval, question answering, and summarization confirm both the theoretical analysis and the conditions that favor multi-agent chunking. By exploring the accelerated decay of model fidelity with input length, we also explain why, for large inputs, a weaker model configured with chunk-based processing can surpass a more advanced model like GPT4o applied in a single shot. Overall, we present a principled understanding framework and our results highlight a direct pathway to handling long contexts in LLMs with carefully managed chunking and aggregator strategies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_16411 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework Xu, Zhen Zhu, Shang Wang, Jue Wang, Junlin Athiwaratkun, Ben Wang, Chi Zou, James Zhang, Ce Computation and Language Machine Learning We investigate the challenge of applying Large Language Models (LLMs) to long texts. We propose a theoretical framework that distinguishes the failure modes of long context tasks into three categories: cross-chunk dependence (task noise), confusion that grows with context size (model noise), and the imperfect integration of partial results (aggregator noise). Under this view, we analyze when it is effective to use multi-agent chunking, i.e., dividing a lengthy sequence into smaller chunks and aggregating the processed results of each chunk. Our experiments on tasks such as retrieval, question answering, and summarization confirm both the theoretical analysis and the conditions that favor multi-agent chunking. By exploring the accelerated decay of model fidelity with input length, we also explain why, for large inputs, a weaker model configured with chunk-based processing can surpass a more advanced model like GPT4o applied in a single shot. Overall, we present a principled understanding framework and our results highlight a direct pathway to handling long contexts in LLMs with carefully managed chunking and aggregator strategies. |
| title | When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2506.16411 |