From Facts to Conclusions : Integrating Deductive Reasoning in Retrieval-Augmented LLMs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908720332537856 |
|---|---|
| author | Mishra, Shubham Jain, Samyek Mehrishi, Gorang Tiwari, Shiv Sharma, Harsh Narang, Pratik Kumar, Dhruv |
| author_facet | Mishra, Shubham Jain, Samyek Mehrishi, Gorang Tiwari, Shiv Sharma, Harsh Narang, Pratik Kumar, Dhruv |
| contents | Retrieval-Augmented Generation (RAG) grounds large language models (LLMs) in external evidence, but fails when retrieved sources conflict or contain outdated or subjective information. Prior work address these issues independently but lack unified reasoning supervision. We propose a reasoning-trace-augmented RAG framework that adds structured, interpretable reasoning across three stages : (1) document-level adjudication, (2) conflict analysis, and (3) grounded synthesis, producing citation-linked answers or justified refusals. A Conflict-Aware Trust-Score (CATS) pipeline is introduced which evaluates groundedness, factual correctness, refusal accuracy, and conflict-behavior alignment using an LLM-as-a-Judge. Our 539-query reasoning dataset and evaluation pipeline establish a foundation for conflict-aware, interpretable RAG systems. Experimental results demonstrate substantial gains over baselines, most notably with Qwen, where Supervised Fine-Tuning improved End-to-End answer correctness from 0.069 to 0.883 and behavioral adherence from 0.074 to 0.722. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_16795 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From Facts to Conclusions : Integrating Deductive Reasoning in Retrieval-Augmented LLMs Mishra, Shubham Jain, Samyek Mehrishi, Gorang Tiwari, Shiv Sharma, Harsh Narang, Pratik Kumar, Dhruv Computation and Language Artificial Intelligence Computers and Society Information Retrieval Retrieval-Augmented Generation (RAG) grounds large language models (LLMs) in external evidence, but fails when retrieved sources conflict or contain outdated or subjective information. Prior work address these issues independently but lack unified reasoning supervision. We propose a reasoning-trace-augmented RAG framework that adds structured, interpretable reasoning across three stages : (1) document-level adjudication, (2) conflict analysis, and (3) grounded synthesis, producing citation-linked answers or justified refusals. A Conflict-Aware Trust-Score (CATS) pipeline is introduced which evaluates groundedness, factual correctness, refusal accuracy, and conflict-behavior alignment using an LLM-as-a-Judge. Our 539-query reasoning dataset and evaluation pipeline establish a foundation for conflict-aware, interpretable RAG systems. Experimental results demonstrate substantial gains over baselines, most notably with Qwen, where Supervised Fine-Tuning improved End-to-End answer correctness from 0.069 to 0.883 and behavioral adherence from 0.074 to 0.722. |
| title | From Facts to Conclusions : Integrating Deductive Reasoning in Retrieval-Augmented LLMs |
| topic | Computation and Language Artificial Intelligence Computers and Society Information Retrieval |
| url | https://arxiv.org/abs/2512.16795 |