Opening the Black Box: A Survey on the Mechanisms of Multi-Step Reasoning in Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914268428894208 |
|---|---|
| author | Pan, Liangming Liang, Jason Ye, Jiaran Yang, Minglai Lu, Xinyuan Zhu, Fengbin |
| author_facet | Pan, Liangming Liang, Jason Ye, Jiaran Yang, Minglai Lu, Xinyuan Zhu, Fengbin |
| contents | Large Language Models (LLMs) have demonstrated remarkable abilities to solve problems requiring multiple reasoning steps, yet the internal mechanisms enabling such capabilities remain elusive. Unlike existing surveys that primarily focus on engineering methods to enhance performance, this survey provides a comprehensive overview of the mechanisms underlying LLM multi-step reasoning. We organize the survey around a conceptual framework comprising seven interconnected research questions, from how LLMs execute implicit multi-hop reasoning within hidden activations to how verbalized explicit reasoning remodels the internal computation. Finally, we highlight five research directions for future mechanistic studies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_14270 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Opening the Black Box: A Survey on the Mechanisms of Multi-Step Reasoning in Large Language Models Pan, Liangming Liang, Jason Ye, Jiaran Yang, Minglai Lu, Xinyuan Zhu, Fengbin Computation and Language Artificial Intelligence Large Language Models (LLMs) have demonstrated remarkable abilities to solve problems requiring multiple reasoning steps, yet the internal mechanisms enabling such capabilities remain elusive. Unlike existing surveys that primarily focus on engineering methods to enhance performance, this survey provides a comprehensive overview of the mechanisms underlying LLM multi-step reasoning. We organize the survey around a conceptual framework comprising seven interconnected research questions, from how LLMs execute implicit multi-hop reasoning within hidden activations to how verbalized explicit reasoning remodels the internal computation. Finally, we highlight five research directions for future mechanistic studies. |
| title | Opening the Black Box: A Survey on the Mechanisms of Multi-Step Reasoning in Large Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2601.14270 |