Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.14140 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914476196888576 |
|---|---|
| author | Motwani, Sumeet Ramesh Nichols, Daniel London, Charles Li, Peggy Pizzati, Fabio Blake, Acer Hammoud, Hasan McDonald, Tavish Naik, Akshat Ivanova, Alesia Baskaran, Vignesh Laptev, Ivan Glatt, Ruben Ben-Nun, Tal Torr, Philip Jaques, Natasha Prabhu, Ameya Bartoldson, Brian Kailkhura, Bhavya de Witt, Christian Schroeder |
| author_facet | Motwani, Sumeet Ramesh Nichols, Daniel London, Charles Li, Peggy Pizzati, Fabio Blake, Acer Hammoud, Hasan McDonald, Tavish Naik, Akshat Ivanova, Alesia Baskaran, Vignesh Laptev, Ivan Glatt, Ruben Ben-Nun, Tal Torr, Philip Jaques, Natasha Prabhu, Ameya Bartoldson, Brian Kailkhura, Bhavya de Witt, Christian Schroeder |
| contents | As language models are increasingly deployed for complex autonomous tasks, their ability to reason accurately over longer horizons becomes critical. An essential component of this ability is planning and managing a long, complex chain-of-thought (CoT). We introduce LongCoT, a scalable benchmark of 2,500 expert-designed problems spanning chemistry, mathematics, computer science, chess, and logic to isolate and directly measure the long-horizon CoT reasoning capabilities of frontier models. Problems consist of a short input with a verifiable answer; solving them requires navigating a graph of interdependent steps that span tens to hundreds of thousands of reasoning tokens. Each local step is individually tractable for frontier models, so failures reflect long-horizon reasoning limitations. At release, the best models achieve <10% accuracy (GPT 5.2: 9.8%; Gemini 3 Pro: 6.1%) on LongCoT, revealing a substantial gap in current capabilities. Overall, LongCoT provides a rigorous measure of long-horizon reasoning, tracking the ability of frontier models to reason reliably over extended periods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_14140 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | LongCoT: Benchmarking Long-Horizon Chain-of-Thought Reasoning Motwani, Sumeet Ramesh Nichols, Daniel London, Charles Li, Peggy Pizzati, Fabio Blake, Acer Hammoud, Hasan McDonald, Tavish Naik, Akshat Ivanova, Alesia Baskaran, Vignesh Laptev, Ivan Glatt, Ruben Ben-Nun, Tal Torr, Philip Jaques, Natasha Prabhu, Ameya Bartoldson, Brian Kailkhura, Bhavya de Witt, Christian Schroeder Machine Learning Artificial Intelligence As language models are increasingly deployed for complex autonomous tasks, their ability to reason accurately over longer horizons becomes critical. An essential component of this ability is planning and managing a long, complex chain-of-thought (CoT). We introduce LongCoT, a scalable benchmark of 2,500 expert-designed problems spanning chemistry, mathematics, computer science, chess, and logic to isolate and directly measure the long-horizon CoT reasoning capabilities of frontier models. Problems consist of a short input with a verifiable answer; solving them requires navigating a graph of interdependent steps that span tens to hundreds of thousands of reasoning tokens. Each local step is individually tractable for frontier models, so failures reflect long-horizon reasoning limitations. At release, the best models achieve <10% accuracy (GPT 5.2: 9.8%; Gemini 3 Pro: 6.1%) on LongCoT, revealing a substantial gap in current capabilities. Overall, LongCoT provides a rigorous measure of long-horizon reasoning, tracking the ability of frontier models to reason reliably over extended periods. |
| title | LongCoT: Benchmarking Long-Horizon Chain-of-Thought Reasoning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2604.14140 |