Beyond Black-Box Interventions: Latent Probing for Faithful Retrieval-Augmented Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914474763485184 |
|---|---|
| author | Gao, Linfeng Zhang, Qinggang Bi, Baolong Zeng, Bo Yuan, Zheng Chen, Zerui Wei, Zhimin Liu, Shenghua Xu, Linlong Wang, Longyue Luo, Weihua Su, Jinsong |
| author_facet | Gao, Linfeng Zhang, Qinggang Bi, Baolong Zeng, Bo Yuan, Zheng Chen, Zerui Wei, Zhimin Liu, Shenghua Xu, Linlong Wang, Longyue Luo, Weihua Su, Jinsong |
| contents | Retrieval-Augmented Generation (RAG) systems often fail to maintain contextual faithfulness, generating responses that conflict with the provided context or fail to fully leverage the provided evidence. Existing methods attempt to improve faithfulness through external interventions, such as specialized prompting, decoding-based calibration, or preference optimization. However, since these approaches treat the LLM as a black box, they lack a reliable mechanism to assess when and why knowledge conflicts occur. Consequently, they tend to be brittle, data-intensive, and agnostic to the model's internal reasoning process. In this paper, we move beyond black-box interventions to analyze the model's internal reasoning process. We discover that conflicting and aligned knowledge states are linearly separable in the model's latent space, and contextual noise systematically increases the entropy of these representations. Based on these findings, we propose ProbeRAG, a novel framework for faithful RAG that operates in three stages: (i) fine-grained knowledge pruning to filter irrelevant context, (ii) latent conflict probing to identify hard conflicts in the model's latent space, and (iii) conflict-aware attention to modulate attention heads toward faithful context integration. Extensive experiments demonstrate that ProbeRAG substantially improves both accuracy and contextual faithfulness. The related resources are available at https://github.com/LinfengGao/ProbeRAG. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_12460 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Beyond Black-Box Interventions: Latent Probing for Faithful Retrieval-Augmented Generation Gao, Linfeng Zhang, Qinggang Bi, Baolong Zeng, Bo Yuan, Zheng Chen, Zerui Wei, Zhimin Liu, Shenghua Xu, Linlong Wang, Longyue Luo, Weihua Su, Jinsong Computation and Language Retrieval-Augmented Generation (RAG) systems often fail to maintain contextual faithfulness, generating responses that conflict with the provided context or fail to fully leverage the provided evidence. Existing methods attempt to improve faithfulness through external interventions, such as specialized prompting, decoding-based calibration, or preference optimization. However, since these approaches treat the LLM as a black box, they lack a reliable mechanism to assess when and why knowledge conflicts occur. Consequently, they tend to be brittle, data-intensive, and agnostic to the model's internal reasoning process. In this paper, we move beyond black-box interventions to analyze the model's internal reasoning process. We discover that conflicting and aligned knowledge states are linearly separable in the model's latent space, and contextual noise systematically increases the entropy of these representations. Based on these findings, we propose ProbeRAG, a novel framework for faithful RAG that operates in three stages: (i) fine-grained knowledge pruning to filter irrelevant context, (ii) latent conflict probing to identify hard conflicts in the model's latent space, and (iii) conflict-aware attention to modulate attention heads toward faithful context integration. Extensive experiments demonstrate that ProbeRAG substantially improves both accuracy and contextual faithfulness. The related resources are available at https://github.com/LinfengGao/ProbeRAG. |
| title | Beyond Black-Box Interventions: Latent Probing for Faithful Retrieval-Augmented Generation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.12460 |