Learning to Seek Evidence: A Verifiable Reasoning Agent with Causal Faithfulness Analysis
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918183475085312 |
|---|---|
| author | Huang, Yuhang Lin, Zekai Zhong, Fan Liu, Lei |
| author_facet | Huang, Yuhang Lin, Zekai Zhong, Fan Liu, Lei |
| contents | Explanations for AI models in high-stakes domains like medicine often lack verifiability, which can hinder trust. To address this, we propose an interactive agent that produces explanations through an auditable sequence of actions. The agent learns a policy to strategically seek external visual evidence to support its diagnostic reasoning. This policy is optimized using reinforcement learning, resulting in a model that is both efficient and generalizable. Our experiments show that this action-based reasoning process significantly improves calibrated accuracy, reducing the Brier score by 18\% compared to a non-interactive baseline. To validate the faithfulness of the agent's explanations, we introduce a causal intervention method. By masking the visual evidence the agent chooses to use, we observe a measurable degradation in its performance ($Δ$Brier=+0.029), confirming that the evidence is integral to its decision-making process. Our work provides a practical framework for building AI systems with verifiable and faithful reasoning capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_01425 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Learning to Seek Evidence: A Verifiable Reasoning Agent with Causal Faithfulness Analysis Huang, Yuhang Lin, Zekai Zhong, Fan Liu, Lei Artificial Intelligence Computer Vision and Pattern Recognition I.2.6; I.2.10 Explanations for AI models in high-stakes domains like medicine often lack verifiability, which can hinder trust. To address this, we propose an interactive agent that produces explanations through an auditable sequence of actions. The agent learns a policy to strategically seek external visual evidence to support its diagnostic reasoning. This policy is optimized using reinforcement learning, resulting in a model that is both efficient and generalizable. Our experiments show that this action-based reasoning process significantly improves calibrated accuracy, reducing the Brier score by 18\% compared to a non-interactive baseline. To validate the faithfulness of the agent's explanations, we introduce a causal intervention method. By masking the visual evidence the agent chooses to use, we observe a measurable degradation in its performance ($Δ$Brier=+0.029), confirming that the evidence is integral to its decision-making process. Our work provides a practical framework for building AI systems with verifiable and faithful reasoning capabilities. |
| title | Learning to Seek Evidence: A Verifiable Reasoning Agent with Causal Faithfulness Analysis |
| topic | Artificial Intelligence Computer Vision and Pattern Recognition I.2.6; I.2.10 |
| url | https://arxiv.org/abs/2511.01425 |