HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Auto-Encoders
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908975022211072 |
|---|---|
| author | Chen, Boshui Fan, Zhaoxin Wang, Ke Leng, Zhiying Wu, Faguo Zheng, Hongwei Sun, Yifan Wu, Wenjun |
| author_facet | Chen, Boshui Fan, Zhaoxin Wang, Ke Leng, Zhiying Wu, Faguo Zheng, Hongwei Sun, Yifan Wu, Wenjun |
| contents | Large Language Models (LLMs) are powerful and widely adopted, but their practical impact is limited by the well-known hallucination phenomenon. While recent hallucination detection methods have made notable progress, we find most of them overlook the dynamic nature and underlying mechanisms of it. To address this gap, we propose HalluSAE, a phase transition-inspired framework that models hallucination as a critical shift in the model's latent dynamics. By modeling the generation process as a trajectory through a potential energy landscape, HalluSAE identifies critical transition zones and attributes factual errors to specific high-energy sparse features. Our approach consists of three stages: (1) Potential Energy Empowered Phase Zone Localization via sparse autoencoders and a geometric potential energy metric; (2) Hallucination-related Sparse Feature Attribution using contrastive logit attribution; and (3) Probing-based Causal Hallucination Detection through linear probes on disentangled features. Extensive experiments on Gemma-2-9B demonstrate that HalluSAE achieves state-of-the-art hallucination detection performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_16430 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Auto-Encoders Chen, Boshui Fan, Zhaoxin Wang, Ke Leng, Zhiying Wu, Faguo Zheng, Hongwei Sun, Yifan Wu, Wenjun Computation and Language Artificial Intelligence Large Language Models (LLMs) are powerful and widely adopted, but their practical impact is limited by the well-known hallucination phenomenon. While recent hallucination detection methods have made notable progress, we find most of them overlook the dynamic nature and underlying mechanisms of it. To address this gap, we propose HalluSAE, a phase transition-inspired framework that models hallucination as a critical shift in the model's latent dynamics. By modeling the generation process as a trajectory through a potential energy landscape, HalluSAE identifies critical transition zones and attributes factual errors to specific high-energy sparse features. Our approach consists of three stages: (1) Potential Energy Empowered Phase Zone Localization via sparse autoencoders and a geometric potential energy metric; (2) Hallucination-related Sparse Feature Attribution using contrastive logit attribution; and (3) Probing-based Causal Hallucination Detection through linear probes on disentangled features. Extensive experiments on Gemma-2-9B demonstrate that HalluSAE achieves state-of-the-art hallucination detection performance. |
| title | HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Auto-Encoders |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2604.16430 |