When Do Hallucinations Arise? A Graph Perspective on the Evolution of Path Reuse and Path Compression
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913004701876224 |
|---|---|
| author | Dai, Xinnan Yang, Kai Luo, Cheng Zeng, Shenglai Guo, Kai Tang, Jiliang |
| author_facet | Dai, Xinnan Yang, Kai Luo, Cheng Zeng, Shenglai Guo, Kai Tang, Jiliang |
| contents | Reasoning hallucinations in large language models (LLMs) often appear as fluent yet unsupported conclusions that violate either the given context or underlying factual knowledge. Although such failures are widely observed, the mechanisms by which decoder-only Transformers produce them remain poorly understood. We model next-token prediction as a graph search process over an underlying graph, where entities correspond to nodes and learned transitions form edges. From this perspective, contextual reasoning is a constrained search over a sampled subgraph (intrinsic reasoning), while context-free queries rely on memorized structures in the underlying graph (extrinsic reasoning). We show that reasoning hallucinations arise from two fundamental mechanisms: \textbf{Path Reuse}, where memorized knowledge overrides contextual constraints during early training, and \textbf{Path Compression}, where frequently traversed multi-step paths collapse into shortcut edges in later training. Together, these mechanisms provide a unified explanation for reasoning hallucinations in LLMs and connected to well-known behaviors observed in downstream applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_03557 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | When Do Hallucinations Arise? A Graph Perspective on the Evolution of Path Reuse and Path Compression Dai, Xinnan Yang, Kai Luo, Cheng Zeng, Shenglai Guo, Kai Tang, Jiliang Artificial Intelligence Reasoning hallucinations in large language models (LLMs) often appear as fluent yet unsupported conclusions that violate either the given context or underlying factual knowledge. Although such failures are widely observed, the mechanisms by which decoder-only Transformers produce them remain poorly understood. We model next-token prediction as a graph search process over an underlying graph, where entities correspond to nodes and learned transitions form edges. From this perspective, contextual reasoning is a constrained search over a sampled subgraph (intrinsic reasoning), while context-free queries rely on memorized structures in the underlying graph (extrinsic reasoning). We show that reasoning hallucinations arise from two fundamental mechanisms: \textbf{Path Reuse}, where memorized knowledge overrides contextual constraints during early training, and \textbf{Path Compression}, where frequently traversed multi-step paths collapse into shortcut edges in later training. Together, these mechanisms provide a unified explanation for reasoning hallucinations in LLMs and connected to well-known behaviors observed in downstream applications. |
| title | When Do Hallucinations Arise? A Graph Perspective on the Evolution of Path Reuse and Path Compression |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2604.03557 |