Dynamic Parallel Tree Search for Efficient LLM Reasoning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910847339593728 |
|---|---|
| author | Ding, Yifu Jiang, Wentao Liu, Shunyu Jing, Yongcheng Guo, Jinyang Wang, Yingjie Zhang, Jing Wang, Zengmao Liu, Ziwei Du, Bo Liu, Xianglong Tao, Dacheng |
| author_facet | Ding, Yifu Jiang, Wentao Liu, Shunyu Jing, Yongcheng Guo, Jinyang Wang, Yingjie Zhang, Jing Wang, Zengmao Liu, Ziwei Du, Bo Liu, Xianglong Tao, Dacheng |
| contents | Tree of Thoughts (ToT) enhances Large Language Model (LLM) reasoning by structuring problem-solving as a spanning tree. However, recent methods focus on search accuracy while overlooking computational efficiency. The challenges of accelerating the ToT lie in the frequent switching of reasoning focus, and the redundant exploration of suboptimal solutions. To alleviate this dilemma, we propose Dynamic Parallel Tree Search (DPTS), a novel parallelism framework that aims to dynamically optimize the reasoning path in inference. It includes the Parallelism Streamline in the generation phase to build up a flexible and adaptive parallelism with arbitrary paths by fine-grained cache management and alignment. Meanwhile, the Search and Transition Mechanism filters potential candidates to dynamically maintain the reasoning focus on more possible solutions and have less redundancy. Experiments on Qwen-2.5 and Llama-3 with Math500 and GSM8K datasets show that DPTS significantly improves efficiency by 2-4x on average while maintaining or even surpassing existing reasoning algorithms in accuracy, making ToT-based reasoning more scalable and computationally efficient. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_16235 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dynamic Parallel Tree Search for Efficient LLM Reasoning Ding, Yifu Jiang, Wentao Liu, Shunyu Jing, Yongcheng Guo, Jinyang Wang, Yingjie Zhang, Jing Wang, Zengmao Liu, Ziwei Du, Bo Liu, Xianglong Tao, Dacheng Artificial Intelligence Tree of Thoughts (ToT) enhances Large Language Model (LLM) reasoning by structuring problem-solving as a spanning tree. However, recent methods focus on search accuracy while overlooking computational efficiency. The challenges of accelerating the ToT lie in the frequent switching of reasoning focus, and the redundant exploration of suboptimal solutions. To alleviate this dilemma, we propose Dynamic Parallel Tree Search (DPTS), a novel parallelism framework that aims to dynamically optimize the reasoning path in inference. It includes the Parallelism Streamline in the generation phase to build up a flexible and adaptive parallelism with arbitrary paths by fine-grained cache management and alignment. Meanwhile, the Search and Transition Mechanism filters potential candidates to dynamically maintain the reasoning focus on more possible solutions and have less redundancy. Experiments on Qwen-2.5 and Llama-3 with Math500 and GSM8K datasets show that DPTS significantly improves efficiency by 2-4x on average while maintaining or even surpassing existing reasoning algorithms in accuracy, making ToT-based reasoning more scalable and computationally efficient. |
| title | Dynamic Parallel Tree Search for Efficient LLM Reasoning |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2502.16235 |