A Decomposition Perspective to Long-context Reasoning for LLMs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910115816275968 |
|---|---|
| author | Xiao, Yanling Xie, Huaibing Zhao, Guoliang Dou, Shihan Wang, Shaolei Liu, Yiting Zheng, Nantao Zhang, Cheng Zhou, Pluto Zhang, Zhisong Liu, Lemao |
| author_facet | Xiao, Yanling Xie, Huaibing Zhao, Guoliang Dou, Shihan Wang, Shaolei Liu, Yiting Zheng, Nantao Zhang, Cheng Zhou, Pluto Zhang, Zhisong Liu, Lemao |
| contents | Long-context reasoning is essential for complex real-world applications, yet remains a significant challenge for Large Language Models (LLMs). Despite the rapid evolution in long-context reasoning, current research often overlooks the internal complexity of the long-context reasoning task itself. In this paper, we move beyond this holistic view and decompose long-context reasoning into a set of fundamental atomic skills, and we then automatically synthesize a suite of pseudo datasets, each explicitly targeting a specific atomic skill. Our empirical analysis confirms that proficiency in these atomic skills is strongly correlated with general long-text reasoning performance. Building on this insight, we employ reinforcement learning on these pseudo datasets to sharpen the model's atomic skills, in the hope of boosting its general long-context reasoning ability. Extensive experiments across multiple benchmarks demonstrate the effectiveness of our approach: it outperforms a strong baseline by an average margin of 7.7\% (improving from 46.3\% to 54.0\%) across Loogle, Loong, LongBench-v2, BrowscompLong, Ruler-qa2, and MRCR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_07981 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A Decomposition Perspective to Long-context Reasoning for LLMs Xiao, Yanling Xie, Huaibing Zhao, Guoliang Dou, Shihan Wang, Shaolei Liu, Yiting Zheng, Nantao Zhang, Cheng Zhou, Pluto Zhang, Zhisong Liu, Lemao Computation and Language Artificial Intelligence Machine Learning Long-context reasoning is essential for complex real-world applications, yet remains a significant challenge for Large Language Models (LLMs). Despite the rapid evolution in long-context reasoning, current research often overlooks the internal complexity of the long-context reasoning task itself. In this paper, we move beyond this holistic view and decompose long-context reasoning into a set of fundamental atomic skills, and we then automatically synthesize a suite of pseudo datasets, each explicitly targeting a specific atomic skill. Our empirical analysis confirms that proficiency in these atomic skills is strongly correlated with general long-text reasoning performance. Building on this insight, we employ reinforcement learning on these pseudo datasets to sharpen the model's atomic skills, in the hope of boosting its general long-context reasoning ability. Extensive experiments across multiple benchmarks demonstrate the effectiveness of our approach: it outperforms a strong baseline by an average margin of 7.7\% (improving from 46.3\% to 54.0\%) across Loogle, Loong, LongBench-v2, BrowscompLong, Ruler-qa2, and MRCR. |
| title | A Decomposition Perspective to Long-context Reasoning for LLMs |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2604.07981 |