Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.12881 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910749530521600 |
|---|---|
| author | Jiang, Jinhao Chen, Jiayi Li, Junyi Ren, Ruiyang Wang, Shijie Zhao, Wayne Xin Song, Yang Zhang, Tao |
| author_facet | Jiang, Jinhao Chen, Jiayi Li, Junyi Ren, Ruiyang Wang, Shijie Zhao, Wayne Xin Song, Yang Zhang, Tao |
| contents | Existing large language models (LLMs) show exceptional problem-solving capabilities but might struggle with complex reasoning tasks. Despite the successes of chain-of-thought and tree-based search methods, they mainly depend on the internal knowledge of LLMs to search over intermediate reasoning steps, limited to dealing with simple tasks involving fewer reasoning steps. In this paper, we propose \textbf{RAG-Star}, a novel RAG approach that integrates the retrieved information to guide the tree-based deliberative reasoning process that relies on the inherent knowledge of LLMs. By leveraging Monte Carlo Tree Search, RAG-Star iteratively plans intermediate sub-queries and answers for reasoning based on the LLM itself. To consolidate internal and external knowledge, we propose an retrieval-augmented verification that utilizes query- and answer-aware reward modeling to provide feedback for the inherent reasoning of LLMs. Our experiments involving Llama-3.1-8B-Instruct and GPT-4o demonstrate that RAG-Star significantly outperforms previous RAG and reasoning methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_12881 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement Jiang, Jinhao Chen, Jiayi Li, Junyi Ren, Ruiyang Wang, Shijie Zhao, Wayne Xin Song, Yang Zhang, Tao Computation and Language Artificial Intelligence Existing large language models (LLMs) show exceptional problem-solving capabilities but might struggle with complex reasoning tasks. Despite the successes of chain-of-thought and tree-based search methods, they mainly depend on the internal knowledge of LLMs to search over intermediate reasoning steps, limited to dealing with simple tasks involving fewer reasoning steps. In this paper, we propose \textbf{RAG-Star}, a novel RAG approach that integrates the retrieved information to guide the tree-based deliberative reasoning process that relies on the inherent knowledge of LLMs. By leveraging Monte Carlo Tree Search, RAG-Star iteratively plans intermediate sub-queries and answers for reasoning based on the LLM itself. To consolidate internal and external knowledge, we propose an retrieval-augmented verification that utilizes query- and answer-aware reward modeling to provide feedback for the inherent reasoning of LLMs. Our experiments involving Llama-3.1-8B-Instruct and GPT-4o demonstrate that RAG-Star significantly outperforms previous RAG and reasoning methods. |
| title | RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2412.12881 |