Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909451730026496 |
|---|---|
| author | Xiang, Violet Snell, Charlie Gandhi, Kanishk Albalak, Alon Singh, Anikait Blagden, Chase Phung, Duy Rafailov, Rafael Lile, Nathan Mahan, Dakota Castricato, Louis Franken, Jan-Philipp Haber, Nick Finn, Chelsea |
| author_facet | Xiang, Violet Snell, Charlie Gandhi, Kanishk Albalak, Alon Singh, Anikait Blagden, Chase Phung, Duy Rafailov, Rafael Lile, Nathan Mahan, Dakota Castricato, Louis Franken, Jan-Philipp Haber, Nick Finn, Chelsea |
| contents | We propose a novel framework, Meta Chain-of-Thought (Meta-CoT), which extends traditional Chain-of-Thought (CoT) by explicitly modeling the underlying reasoning required to arrive at a particular CoT. We present empirical evidence from state-of-the-art models exhibiting behaviors consistent with in-context search, and explore methods for producing Meta-CoT via process supervision, synthetic data generation, and search algorithms. Finally, we outline a concrete pipeline for training a model to produce Meta-CoTs, incorporating instruction tuning with linearized search traces and reinforcement learning post-training. Finally, we discuss open research questions, including scaling laws, verifier roles, and the potential for discovering novel reasoning algorithms. This work provides a theoretical and practical roadmap to enable Meta-CoT in LLMs, paving the way for more powerful and human-like reasoning in artificial intelligence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_04682 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought Xiang, Violet Snell, Charlie Gandhi, Kanishk Albalak, Alon Singh, Anikait Blagden, Chase Phung, Duy Rafailov, Rafael Lile, Nathan Mahan, Dakota Castricato, Louis Franken, Jan-Philipp Haber, Nick Finn, Chelsea Artificial Intelligence Computation and Language We propose a novel framework, Meta Chain-of-Thought (Meta-CoT), which extends traditional Chain-of-Thought (CoT) by explicitly modeling the underlying reasoning required to arrive at a particular CoT. We present empirical evidence from state-of-the-art models exhibiting behaviors consistent with in-context search, and explore methods for producing Meta-CoT via process supervision, synthetic data generation, and search algorithms. Finally, we outline a concrete pipeline for training a model to produce Meta-CoTs, incorporating instruction tuning with linearized search traces and reinforcement learning post-training. Finally, we discuss open research questions, including scaling laws, verifier roles, and the potential for discovering novel reasoning algorithms. This work provides a theoretical and practical roadmap to enable Meta-CoT in LLMs, paving the way for more powerful and human-like reasoning in artificial intelligence. |
| title | Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2501.04682 |