Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.10858 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929718718103552 |
|---|---|
| author | Wu, Zongqian Li, Tianyu Xu, Baoduo Yang, Jiaying Zhan, Mengmeng Zhu, Xiaofeng Feng, Lei |
| author_facet | Wu, Zongqian Li, Tianyu Xu, Baoduo Yang, Jiaying Zhan, Mengmeng Zhu, Xiaofeng Feng, Lei |
| contents | Deep iterative chain-of-thought (CoT) reasoning enables LLMs to tackle complex tasks by progressively activating relevant pre-trained knowledge. However, it faces challenges in ensuring continual improvement and determining a stopping criterion. In this paper, we investigate whether the relevant knowledge that contributes directly to solving the given question can be activated from the initial reasoning path, thus circumventing the need for iterative refinement. Our experiments reveal that increasing the diversity of initial reasoning paths can achieve comparable or superior performance, a concept we term \textit{breadth reasoning}. However, existing breadth reasoning approaches, such as self-consistency, offer limited diversity. To address this limitation, we propose a simple yet effective method that enhances reasoning breadth by integrating contextual exploration with reduced sampling randomness. Extensive experiments demonstrate that our approach significantly outperforms deep iterative reasoning. Our code is provided in https://github.com/zongqianwu/breadth. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_10858 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Is Depth All You Need? An Exploration of Iterative Reasoning in LLMs Wu, Zongqian Li, Tianyu Xu, Baoduo Yang, Jiaying Zhan, Mengmeng Zhu, Xiaofeng Feng, Lei Artificial Intelligence Computation and Language Deep iterative chain-of-thought (CoT) reasoning enables LLMs to tackle complex tasks by progressively activating relevant pre-trained knowledge. However, it faces challenges in ensuring continual improvement and determining a stopping criterion. In this paper, we investigate whether the relevant knowledge that contributes directly to solving the given question can be activated from the initial reasoning path, thus circumventing the need for iterative refinement. Our experiments reveal that increasing the diversity of initial reasoning paths can achieve comparable or superior performance, a concept we term \textit{breadth reasoning}. However, existing breadth reasoning approaches, such as self-consistency, offer limited diversity. To address this limitation, we propose a simple yet effective method that enhances reasoning breadth by integrating contextual exploration with reduced sampling randomness. Extensive experiments demonstrate that our approach significantly outperforms deep iterative reasoning. Our code is provided in https://github.com/zongqianwu/breadth. |
| title | Is Depth All You Need? An Exploration of Iterative Reasoning in LLMs |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2502.10858 |