Reasoning emerges from constrained inference manifolds in large language models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910202429702144 |
|---|---|
| author | Ma, Yanbiao Luo, Fei Zhang, Linfeng Zhao, Chuangxin Wang, Mingxuan Wu, Yinan Qian, Zhe Lu, Yang Chen, Long Cao, Zhao Hao, Xiaoshuai Wen, Ji-Rong Han, Jungong |
| author_facet | Ma, Yanbiao Luo, Fei Zhang, Linfeng Zhao, Chuangxin Wang, Mingxuan Wu, Yinan Qian, Zhe Lu, Yang Chen, Long Cao, Zhao Hao, Xiaoshuai Wen, Ji-Rong Han, Jungong |
| contents | Reasoning in large language models is predominantly evaluated through labeled benchmarks, conflating task performance with the quality of internal inference. Here we study reasoning as an intrinsic dynamical process by examining the evolution of internal representations during inference. We find that inference-time dynamics consistently self-organize into low-dimensional manifolds embedded within high-dimensional representation spaces. we find that such geometric compression, although pervasive, is not sufficient for stable or reliable reasoning. Instead, effective reasoning dynamics emerge within a constrained structural regime characterized by three conditions: adequate representational expressivity, spontaneous manifold compression, and preservation of non-degenerate information volume within the compressed subspace. Models outside this regime exhibit characteristic pathological inference dynamics. Based on these insights, we introduce a unified, label-free diagnostic computed solely from internal dynamics. These findings suggest that reasoning in LLMs is fundamentally governed by geometric and informational constraints, offering a complementary framework to benchmark-centric assessment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_08142 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Reasoning emerges from constrained inference manifolds in large language models Ma, Yanbiao Luo, Fei Zhang, Linfeng Zhao, Chuangxin Wang, Mingxuan Wu, Yinan Qian, Zhe Lu, Yang Chen, Long Cao, Zhao Hao, Xiaoshuai Wen, Ji-Rong Han, Jungong Machine Learning Computation and Language Computer Vision and Pattern Recognition Reasoning in large language models is predominantly evaluated through labeled benchmarks, conflating task performance with the quality of internal inference. Here we study reasoning as an intrinsic dynamical process by examining the evolution of internal representations during inference. We find that inference-time dynamics consistently self-organize into low-dimensional manifolds embedded within high-dimensional representation spaces. we find that such geometric compression, although pervasive, is not sufficient for stable or reliable reasoning. Instead, effective reasoning dynamics emerge within a constrained structural regime characterized by three conditions: adequate representational expressivity, spontaneous manifold compression, and preservation of non-degenerate information volume within the compressed subspace. Models outside this regime exhibit characteristic pathological inference dynamics. Based on these insights, we introduce a unified, label-free diagnostic computed solely from internal dynamics. These findings suggest that reasoning in LLMs is fundamentally governed by geometric and informational constraints, offering a complementary framework to benchmark-centric assessment. |
| title | Reasoning emerges from constrained inference manifolds in large language models |
| topic | Machine Learning Computation and Language Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.08142 |