Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.17088 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918141430333440 |
|---|---|
| author | Cheng, Jiahao Su, Tiancheng Yuan, Jia He, Guoxiu Liu, Jiawei Tao, Xinqi Xie, Jingwen Li, Huaxia |
| author_facet | Cheng, Jiahao Su, Tiancheng Yuan, Jia He, Guoxiu Liu, Jiawei Tao, Xinqi Xie, Jingwen Li, Huaxia |
| contents | Large Language Models (LLMs) often exhibit \textit{hallucinations}, generating factually incorrect or semantically irrelevant content in response to prompts. Chain-of-Thought (CoT) prompting can mitigate hallucinations by encouraging step-by-step reasoning, but its impact on hallucination detection remains underexplored. To bridge this gap, we conduct a systematic empirical evaluation. We begin with a pilot experiment, revealing that CoT reasoning significantly affects the LLM's internal states and token probability distributions. Building on this, we evaluate the impact of various CoT prompting methods on mainstream hallucination detection methods across both instruction-tuned and reasoning-oriented LLMs. Specifically, we examine three key dimensions: changes in hallucination score distributions, variations in detection accuracy, and shifts in detection confidence. Our findings show that while CoT prompting helps reduce hallucination frequency, it also tends to obscure critical signals used for detection, impairing the effectiveness of various detection methods. Our study highlights an overlooked trade-off in the use of reasoning. Code is publicly available at: https://github.com/ECNU-Text-Computing/cot-hallu-detect . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_17088 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Chain-of-Thought Prompting Obscures Hallucination Cues in Large Language Models: An Empirical Evaluation Cheng, Jiahao Su, Tiancheng Yuan, Jia He, Guoxiu Liu, Jiawei Tao, Xinqi Xie, Jingwen Li, Huaxia Computation and Language Large Language Models (LLMs) often exhibit \textit{hallucinations}, generating factually incorrect or semantically irrelevant content in response to prompts. Chain-of-Thought (CoT) prompting can mitigate hallucinations by encouraging step-by-step reasoning, but its impact on hallucination detection remains underexplored. To bridge this gap, we conduct a systematic empirical evaluation. We begin with a pilot experiment, revealing that CoT reasoning significantly affects the LLM's internal states and token probability distributions. Building on this, we evaluate the impact of various CoT prompting methods on mainstream hallucination detection methods across both instruction-tuned and reasoning-oriented LLMs. Specifically, we examine three key dimensions: changes in hallucination score distributions, variations in detection accuracy, and shifts in detection confidence. Our findings show that while CoT prompting helps reduce hallucination frequency, it also tends to obscure critical signals used for detection, impairing the effectiveness of various detection methods. Our study highlights an overlooked trade-off in the use of reasoning. Code is publicly available at: https://github.com/ECNU-Text-Computing/cot-hallu-detect . |
| title | Chain-of-Thought Prompting Obscures Hallucination Cues in Large Language Models: An Empirical Evaluation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2506.17088 |