Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.06203 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915381448278016 |
|---|---|
| author | Zhu, Rui-Jie Peng, Tianhao Cheng, Tianhao Qu, Xingwei Huang, Jinfa Zhu, Dawei Wang, Hao Xue, Kaiwen Zhang, Xuanliang Shan, Yong Cai, Tianle Kergan, Taylor Kembay, Assel Smith, Andrew Lin, Chenghua Nguyen, Binh Pan, Yuqi Chou, Yuhong Cai, Zefan Wu, Zhenhe Zhao, Yongchi Liu, Tianyu Yang, Jian Zhou, Wangchunshu Zheng, Chujie Li, Chongxuan Zhou, Yuyin Li, Zhoujun Zhang, Zhaoxiang Liu, Jiaheng Zhang, Ge Huang, Wenhao Eshraghian, Jason |
| author_facet | Zhu, Rui-Jie Peng, Tianhao Cheng, Tianhao Qu, Xingwei Huang, Jinfa Zhu, Dawei Wang, Hao Xue, Kaiwen Zhang, Xuanliang Shan, Yong Cai, Tianle Kergan, Taylor Kembay, Assel Smith, Andrew Lin, Chenghua Nguyen, Binh Pan, Yuqi Chou, Yuhong Cai, Zefan Wu, Zhenhe Zhao, Yongchi Liu, Tianyu Yang, Jian Zhou, Wangchunshu Zheng, Chujie Li, Chongxuan Zhou, Yuyin Li, Zhoujun Zhang, Zhaoxiang Liu, Jiaheng Zhang, Ge Huang, Wenhao Eshraghian, Jason |
| contents | Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, especially when guided by explicit chain-of-thought (CoT) reasoning that verbalizes intermediate steps. While CoT improves both interpretability and accuracy, its dependence on natural language reasoning limits the model's expressive bandwidth. Latent reasoning tackles this bottleneck by performing multi-step inference entirely in the model's continuous hidden state, eliminating token-level supervision. To advance latent reasoning research, this survey provides a comprehensive overview of the emerging field of latent reasoning. We begin by examining the foundational role of neural network layers as the computational substrate for reasoning, highlighting how hierarchical representations support complex transformations. Next, we explore diverse latent reasoning methodologies, including activation-based recurrence, hidden state propagation, and fine-tuning strategies that compress or internalize explicit reasoning traces. Finally, we discuss advanced paradigms such as infinite-depth latent reasoning via masked diffusion models, which enable globally consistent and reversible reasoning processes. By unifying these perspectives, we aim to clarify the conceptual landscape of latent reasoning and chart future directions for research at the frontier of LLM cognition. An associated GitHub repository collecting the latest papers and repos is available at: https://github.com/multimodal-art-projection/LatentCoT-Horizon/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_06203 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Survey on Latent Reasoning Zhu, Rui-Jie Peng, Tianhao Cheng, Tianhao Qu, Xingwei Huang, Jinfa Zhu, Dawei Wang, Hao Xue, Kaiwen Zhang, Xuanliang Shan, Yong Cai, Tianle Kergan, Taylor Kembay, Assel Smith, Andrew Lin, Chenghua Nguyen, Binh Pan, Yuqi Chou, Yuhong Cai, Zefan Wu, Zhenhe Zhao, Yongchi Liu, Tianyu Yang, Jian Zhou, Wangchunshu Zheng, Chujie Li, Chongxuan Zhou, Yuyin Li, Zhoujun Zhang, Zhaoxiang Liu, Jiaheng Zhang, Ge Huang, Wenhao Eshraghian, Jason Computation and Language Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, especially when guided by explicit chain-of-thought (CoT) reasoning that verbalizes intermediate steps. While CoT improves both interpretability and accuracy, its dependence on natural language reasoning limits the model's expressive bandwidth. Latent reasoning tackles this bottleneck by performing multi-step inference entirely in the model's continuous hidden state, eliminating token-level supervision. To advance latent reasoning research, this survey provides a comprehensive overview of the emerging field of latent reasoning. We begin by examining the foundational role of neural network layers as the computational substrate for reasoning, highlighting how hierarchical representations support complex transformations. Next, we explore diverse latent reasoning methodologies, including activation-based recurrence, hidden state propagation, and fine-tuning strategies that compress or internalize explicit reasoning traces. Finally, we discuss advanced paradigms such as infinite-depth latent reasoning via masked diffusion models, which enable globally consistent and reversible reasoning processes. By unifying these perspectives, we aim to clarify the conceptual landscape of latent reasoning and chart future directions for research at the frontier of LLM cognition. An associated GitHub repository collecting the latest papers and repos is available at: https://github.com/multimodal-art-projection/LatentCoT-Horizon/. |
| title | A Survey on Latent Reasoning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2507.06203 |