Improving Context Fidelity via Native Retrieval-Augmented Reasoning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914041716277248 |
|---|---|
| author | Wang, Suyuchen Wang, Jinlin Wang, Xinyu Li, Shiqi Tang, Xiangru Hong, Sirui Chang, Xiao-Wen Wu, Chenglin Liu, Bang |
| author_facet | Wang, Suyuchen Wang, Jinlin Wang, Xinyu Li, Shiqi Tang, Xiangru Hong, Sirui Chang, Xiao-Wen Wu, Chenglin Liu, Bang |
| contents | Large language models (LLMs) often struggle with context fidelity, producing inconsistent answers when responding to questions based on provided information. Existing approaches either rely on expensive supervised fine-tuning to generate evidence post-answer or train models to perform web searches without necessarily improving utilization of the given context. We propose CARE, a novel native retrieval-augmented reasoning framework that teaches LLMs to explicitly integrate in-context evidence within their reasoning process with the model's own retrieval capabilities. Our method requires limited labeled evidence data while significantly enhancing both retrieval accuracy and answer generation performance through strategically retrieved in-context tokens in the reasoning chain. Extensive experiments on multiple real-world and counterfactual QA benchmarks demonstrate that our approach substantially outperforms supervised fine-tuning, traditional retrieval-augmented generation methods, and external retrieval solutions. This work represents a fundamental advancement in making LLMs more accurate, reliable, and efficient for knowledge-intensive tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_13683 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Improving Context Fidelity via Native Retrieval-Augmented Reasoning Wang, Suyuchen Wang, Jinlin Wang, Xinyu Li, Shiqi Tang, Xiangru Hong, Sirui Chang, Xiao-Wen Wu, Chenglin Liu, Bang Computation and Language Artificial Intelligence Large language models (LLMs) often struggle with context fidelity, producing inconsistent answers when responding to questions based on provided information. Existing approaches either rely on expensive supervised fine-tuning to generate evidence post-answer or train models to perform web searches without necessarily improving utilization of the given context. We propose CARE, a novel native retrieval-augmented reasoning framework that teaches LLMs to explicitly integrate in-context evidence within their reasoning process with the model's own retrieval capabilities. Our method requires limited labeled evidence data while significantly enhancing both retrieval accuracy and answer generation performance through strategically retrieved in-context tokens in the reasoning chain. Extensive experiments on multiple real-world and counterfactual QA benchmarks demonstrate that our approach substantially outperforms supervised fine-tuning, traditional retrieval-augmented generation methods, and external retrieval solutions. This work represents a fundamental advancement in making LLMs more accurate, reliable, and efficient for knowledge-intensive tasks. |
| title | Improving Context Fidelity via Native Retrieval-Augmented Reasoning |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2509.13683 |