Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2506.06020 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910992117530624 |
|---|---|
| author | Zhou, Zeqi Wu, Fang Talaei, Shayan Zhao, Haokai Meixin, Cheng Xu, Tinson Saberi, Amin Choi, Yejin |
| author_facet | Zhou, Zeqi Wu, Fang Talaei, Shayan Zhao, Haokai Meixin, Cheng Xu, Tinson Saberi, Amin Choi, Yejin |
| contents | Large language models frequently encounter conflicts between their parametric knowledge and contextual input, often resulting in factual inconsistencies or hallucinations. We propose Self-Reflective Debate for Contextual Reliability (SR-DCR), a lightweight framework that integrates token-level self-confidence with an asymmetric multi-agent debate to adjudicate such conflicts. A critic, deprived of context, challenges a defender who argues from the given passage; a judge model evaluates the debate and determines the context's reliability. The final answer is selected by combining the verdict with model confidence. Experiments on the ClashEval benchmark demonstrate that SR-DCR consistently enhances robustness to misleading context while maintaining accuracy on trustworthy inputs, outperforming both classical debate and confidence-only baselines with minimal computational overhead. The code is available at https://github.com/smiles724/Self-Reflective-Debates. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_06020 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | When to Trust Context: Self-Reflective Debates for Context Reliability Zhou, Zeqi Wu, Fang Talaei, Shayan Zhao, Haokai Meixin, Cheng Xu, Tinson Saberi, Amin Choi, Yejin Computation and Language Artificial Intelligence Large language models frequently encounter conflicts between their parametric knowledge and contextual input, often resulting in factual inconsistencies or hallucinations. We propose Self-Reflective Debate for Contextual Reliability (SR-DCR), a lightweight framework that integrates token-level self-confidence with an asymmetric multi-agent debate to adjudicate such conflicts. A critic, deprived of context, challenges a defender who argues from the given passage; a judge model evaluates the debate and determines the context's reliability. The final answer is selected by combining the verdict with model confidence. Experiments on the ClashEval benchmark demonstrate that SR-DCR consistently enhances robustness to misleading context while maintaining accuracy on trustworthy inputs, outperforming both classical debate and confidence-only baselines with minimal computational overhead. The code is available at https://github.com/smiles724/Self-Reflective-Debates. |
| title | When to Trust Context: Self-Reflective Debates for Context Reliability |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2506.06020 |