A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910941204971520 |
|---|---|
| author | Shelmanov, Artem Fadeeva, Ekaterina Tsvigun, Akim Tsvigun, Ivan Xie, Zhuohan Kiselev, Igor Daheim, Nico Zhang, Caiqi Vazhentsev, Artem Sachan, Mrinmaya Nakov, Preslav Baldwin, Timothy |
| author_facet | Shelmanov, Artem Fadeeva, Ekaterina Tsvigun, Akim Tsvigun, Ivan Xie, Zhuohan Kiselev, Igor Daheim, Nico Zhang, Caiqi Vazhentsev, Artem Sachan, Mrinmaya Nakov, Preslav Baldwin, Timothy |
| contents | Large Language Models (LLMs) have the tendency to hallucinate, i.e., to sporadically generate false or fabricated information. This presents a major challenge, as hallucinations often appear highly convincing and users generally lack the tools to detect them. Uncertainty quantification (UQ) provides a framework for assessing the reliability of model outputs, aiding in the identification of potential hallucinations. In this work, we introduce pre-trained UQ heads: supervised auxiliary modules for LLMs that substantially enhance their ability to capture uncertainty compared to unsupervised UQ methods. Their strong performance stems from the powerful Transformer architecture in their design and informative features derived from LLM attention maps. Experimental evaluation shows that these heads are highly robust and achieve state-of-the-art performance in claim-level hallucination detection across both in-domain and out-of-domain prompts. Moreover, these modules demonstrate strong generalization to languages they were not explicitly trained on. We pre-train a collection of UQ heads for popular LLM series, including Mistral, Llama, and Gemma 2. We publicly release both the code and the pre-trained heads. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_08200 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs Shelmanov, Artem Fadeeva, Ekaterina Tsvigun, Akim Tsvigun, Ivan Xie, Zhuohan Kiselev, Igor Daheim, Nico Zhang, Caiqi Vazhentsev, Artem Sachan, Mrinmaya Nakov, Preslav Baldwin, Timothy Computation and Language Artificial Intelligence Large Language Models (LLMs) have the tendency to hallucinate, i.e., to sporadically generate false or fabricated information. This presents a major challenge, as hallucinations often appear highly convincing and users generally lack the tools to detect them. Uncertainty quantification (UQ) provides a framework for assessing the reliability of model outputs, aiding in the identification of potential hallucinations. In this work, we introduce pre-trained UQ heads: supervised auxiliary modules for LLMs that substantially enhance their ability to capture uncertainty compared to unsupervised UQ methods. Their strong performance stems from the powerful Transformer architecture in their design and informative features derived from LLM attention maps. Experimental evaluation shows that these heads are highly robust and achieve state-of-the-art performance in claim-level hallucination detection across both in-domain and out-of-domain prompts. Moreover, these modules demonstrate strong generalization to languages they were not explicitly trained on. We pre-train a collection of UQ heads for popular LLM series, including Mistral, Llama, and Gemma 2. We publicly release both the code and the pre-trained heads. |
| title | A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2505.08200 |