Rethinking Reflection in Pre-Training
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912311014326272 |
|---|---|
| author | AI, Essential : Shah, Darsh J Rushton, Peter Singla, Somanshu Parmar, Mohit Smith, Kurt Vanjani, Yash Vaswani, Ashish Chaluvaraju, Adarsh Hojel, Andrew Ma, Andrew Thomas, Anil Polloreno, Anthony Tanwer, Ashish Sibai, Burhan Drak Mansingka, Divya S Shivaprasad, Divya Shah, Ishaan Stratos, Karl Nguyen, Khoi Callahan, Michael Pust, Michael Iyer, Mrinal Monk, Philip Mazarakis, Platon Kapila, Ritvik Srivastava, Saurabh Romanski, Tim |
| author_facet | AI, Essential : Shah, Darsh J Rushton, Peter Singla, Somanshu Parmar, Mohit Smith, Kurt Vanjani, Yash Vaswani, Ashish Chaluvaraju, Adarsh Hojel, Andrew Ma, Andrew Thomas, Anil Polloreno, Anthony Tanwer, Ashish Sibai, Burhan Drak Mansingka, Divya S Shivaprasad, Divya Shah, Ishaan Stratos, Karl Nguyen, Khoi Callahan, Michael Pust, Michael Iyer, Mrinal Monk, Philip Mazarakis, Platon Kapila, Ritvik Srivastava, Saurabh Romanski, Tim |
| contents | A language model's ability to reflect on its own reasoning provides a key advantage for solving complex problems. While most recent research has focused on how this ability develops during reinforcement learning, we show that it actually begins to emerge much earlier - during the model's pre-training. To study this, we introduce deliberate errors into chains-of-thought and test whether the model can still arrive at the correct answer by recognizing and correcting these mistakes. By tracking performance across different stages of pre-training, we observe that this self-correcting ability appears early and improves steadily over time. For instance, an OLMo2-7B model pre-trained on 4 trillion tokens displays self-correction on our six self-reflection tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_04022 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Rethinking Reflection in Pre-Training AI, Essential : Shah, Darsh J Rushton, Peter Singla, Somanshu Parmar, Mohit Smith, Kurt Vanjani, Yash Vaswani, Ashish Chaluvaraju, Adarsh Hojel, Andrew Ma, Andrew Thomas, Anil Polloreno, Anthony Tanwer, Ashish Sibai, Burhan Drak Mansingka, Divya S Shivaprasad, Divya Shah, Ishaan Stratos, Karl Nguyen, Khoi Callahan, Michael Pust, Michael Iyer, Mrinal Monk, Philip Mazarakis, Platon Kapila, Ritvik Srivastava, Saurabh Romanski, Tim Computation and Language Artificial Intelligence A language model's ability to reflect on its own reasoning provides a key advantage for solving complex problems. While most recent research has focused on how this ability develops during reinforcement learning, we show that it actually begins to emerge much earlier - during the model's pre-training. To study this, we introduce deliberate errors into chains-of-thought and test whether the model can still arrive at the correct answer by recognizing and correcting these mistakes. By tracking performance across different stages of pre-training, we observe that this self-correcting ability appears early and improves steadily over time. For instance, an OLMo2-7B model pre-trained on 4 trillion tokens displays self-correction on our six self-reflection tasks. |
| title | Rethinking Reflection in Pre-Training |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2504.04022 |