Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.11912 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914470039650304 |
|---|---|
| author | Huang, Jianhao Zhou, Zhanpeng Xia, Renqiu Mirzasoleiman, Baharan Su, Weijie Huang, Wei |
| author_facet | Huang, Jianhao Zhou, Zhanpeng Xia, Renqiu Mirzasoleiman, Baharan Su, Weijie Huang, Wei |
| contents | While next-token prediction (NTP) has been the standard objective for training language models, it often struggles to capture global structure in reasoning tasks. Multi-token prediction (MTP) has recently emerged as a promising alternative, yet its underlying mechanisms remain poorly understood. In this paper, we study how MTP facilitates reasoning, with a focus on planning. Empirically, we show that MTP consistently outperforms NTP on both synthetic graph path-finding tasks and more realistic reasoning benchmarks, such as Countdown and boolean satisfiability problems. Theoretically, we analyze a simplified two-layer Transformer on a star graph task. We prove that MTP induces a two-stage reverse reasoning process: the model first attends to the end node and then reconstructs the path by tracing intermediate nodes backward. This behavior arises from a gradient decoupling property of MTP, which provides a cleaner training signal compared to NTP. Ultimately, our results highlight how multi-token objectives inherently bias optimization toward robust and interpretable reasoning circuits. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_11912 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | How Transformers Learn to Plan via Multi-Token Prediction Huang, Jianhao Zhou, Zhanpeng Xia, Renqiu Mirzasoleiman, Baharan Su, Weijie Huang, Wei Machine Learning Artificial Intelligence While next-token prediction (NTP) has been the standard objective for training language models, it often struggles to capture global structure in reasoning tasks. Multi-token prediction (MTP) has recently emerged as a promising alternative, yet its underlying mechanisms remain poorly understood. In this paper, we study how MTP facilitates reasoning, with a focus on planning. Empirically, we show that MTP consistently outperforms NTP on both synthetic graph path-finding tasks and more realistic reasoning benchmarks, such as Countdown and boolean satisfiability problems. Theoretically, we analyze a simplified two-layer Transformer on a star graph task. We prove that MTP induces a two-stage reverse reasoning process: the model first attends to the end node and then reconstructs the path by tracing intermediate nodes backward. This behavior arises from a gradient decoupling property of MTP, which provides a cleaner training signal compared to NTP. Ultimately, our results highlight how multi-token objectives inherently bias optimization toward robust and interpretable reasoning circuits. |
| title | How Transformers Learn to Plan via Multi-Token Prediction |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2604.11912 |