Cross-Examiner: Evaluating Consistency of Large Language Model-Generated Explanations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915192546263040 |
|---|---|
| author | Villa, Danielle Chang, Maria Murugesan, Keerthiram Uceda-Sosa, Rosario Ramamurthy, Karthikeyan Natesan |
| author_facet | Villa, Danielle Chang, Maria Murugesan, Keerthiram Uceda-Sosa, Rosario Ramamurthy, Karthikeyan Natesan |
| contents | Large Language Models (LLMs) are often asked to explain their outputs to enhance accuracy and transparency. However, evidence suggests that these explanations can misrepresent the models' true reasoning processes. One effective way to identify inaccuracies or omissions in these explanations is through consistency checking, which typically involves asking follow-up questions. This paper introduces, cross-examiner, a new method for generating follow-up questions based on a model's explanation of an initial question. Our method combines symbolic information extraction with language model-driven question generation, resulting in better follow-up questions than those produced by LLMs alone. Additionally, this approach is more flexible than other methods and can generate a wider variety of follow-up questions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_08815 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Cross-Examiner: Evaluating Consistency of Large Language Model-Generated Explanations Villa, Danielle Chang, Maria Murugesan, Keerthiram Uceda-Sosa, Rosario Ramamurthy, Karthikeyan Natesan Computation and Language Artificial Intelligence Large Language Models (LLMs) are often asked to explain their outputs to enhance accuracy and transparency. However, evidence suggests that these explanations can misrepresent the models' true reasoning processes. One effective way to identify inaccuracies or omissions in these explanations is through consistency checking, which typically involves asking follow-up questions. This paper introduces, cross-examiner, a new method for generating follow-up questions based on a model's explanation of an initial question. Our method combines symbolic information extraction with language model-driven question generation, resulting in better follow-up questions than those produced by LLMs alone. Additionally, this approach is more flexible than other methods and can generate a wider variety of follow-up questions. |
| title | Cross-Examiner: Evaluating Consistency of Large Language Model-Generated Explanations |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.08815 |