Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915590824787968 |
|---|---|
| author | Chen, Xinghao Zhao, Anhao Xia, Heming Lu, Xuan Wang, Hanlin Chen, Yanjun Zhang, Wei Wang, Jian Li, Wenjie Shen, Xiaoyu |
| author_facet | Chen, Xinghao Zhao, Anhao Xia, Heming Lu, Xuan Wang, Hanlin Chen, Yanjun Zhang, Wei Wang, Jian Li, Wenjie Shen, Xiaoyu |
| contents | Large Language Models (LLMs) have shown impressive performance on complex tasks through Chain-of-Thought (CoT) reasoning. However, conventional CoT relies on explicitly verbalized intermediate steps, which constrains its broader applicability, particularly in abstract reasoning tasks beyond language. To address this, there has been growing research interest in \textit{latent CoT reasoning}, where the reasoning process is embedded within latent spaces. By decoupling reasoning from explicit language generation, latent CoT offers the promise of richer cognitive representations and facilitates more flexible, faster inference. This paper aims to present a comprehensive overview of this emerging paradigm and establish a systematic taxonomy. We analyze recent advances in methods, categorizing them from token-wise horizontal approaches to layer-wise vertical strategies. We then provide in-depth discussions of these methods, highlighting their design principles, applications, and remaining challenges. We hope that our survey provides a structured foundation for advancing this promising direction in LLM reasoning. The relevant papers will be regularly updated at https://github.com/EIT-NLP/Awesome-Latent-CoT. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_16782 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning Chen, Xinghao Zhao, Anhao Xia, Heming Lu, Xuan Wang, Hanlin Chen, Yanjun Zhang, Wei Wang, Jian Li, Wenjie Shen, Xiaoyu Computation and Language Large Language Models (LLMs) have shown impressive performance on complex tasks through Chain-of-Thought (CoT) reasoning. However, conventional CoT relies on explicitly verbalized intermediate steps, which constrains its broader applicability, particularly in abstract reasoning tasks beyond language. To address this, there has been growing research interest in \textit{latent CoT reasoning}, where the reasoning process is embedded within latent spaces. By decoupling reasoning from explicit language generation, latent CoT offers the promise of richer cognitive representations and facilitates more flexible, faster inference. This paper aims to present a comprehensive overview of this emerging paradigm and establish a systematic taxonomy. We analyze recent advances in methods, categorizing them from token-wise horizontal approaches to layer-wise vertical strategies. We then provide in-depth discussions of these methods, highlighting their design principles, applications, and remaining challenges. We hope that our survey provides a structured foundation for advancing this promising direction in LLM reasoning. The relevant papers will be regularly updated at https://github.com/EIT-NLP/Awesome-Latent-CoT. |
| title | Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.16782 |