o1-Coder: an o1 Replication for Coding
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916514975711232 |
|---|---|
| author | Zhang, Yuxiang Wu, Shangxi Yang, Yuqi Shu, Jiangming Xiao, Jinlin Kong, Chao Sang, Jitao |
| author_facet | Zhang, Yuxiang Wu, Shangxi Yang, Yuqi Shu, Jiangming Xiao, Jinlin Kong, Chao Sang, Jitao |
| contents | The technical report introduces O1-CODER, an attempt to replicate OpenAI's o1 model with a focus on coding tasks. It integrates reinforcement learning (RL) and Monte Carlo Tree Search (MCTS) to enhance the model's System-2 thinking capabilities. The framework includes training a Test Case Generator (TCG) for standardized code testing, using MCTS to generate code data with reasoning processes, and iteratively fine-tuning the policy model to initially produce pseudocode and then generate the full code. The report also addresses the opportunities and challenges in deploying o1-like models in real-world applications, suggesting transitioning to the System-2 paradigm and highlighting the imperative for world model construction. Updated model progress and experimental results will be reported in subsequent versions. All source code, curated datasets, as well as the derived models are disclosed at https://github.com/ADaM-BJTU/O1-CODER . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_00154 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | o1-Coder: an o1 Replication for Coding Zhang, Yuxiang Wu, Shangxi Yang, Yuqi Shu, Jiangming Xiao, Jinlin Kong, Chao Sang, Jitao Software Engineering Artificial Intelligence The technical report introduces O1-CODER, an attempt to replicate OpenAI's o1 model with a focus on coding tasks. It integrates reinforcement learning (RL) and Monte Carlo Tree Search (MCTS) to enhance the model's System-2 thinking capabilities. The framework includes training a Test Case Generator (TCG) for standardized code testing, using MCTS to generate code data with reasoning processes, and iteratively fine-tuning the policy model to initially produce pseudocode and then generate the full code. The report also addresses the opportunities and challenges in deploying o1-like models in real-world applications, suggesting transitioning to the System-2 paradigm and highlighting the imperative for world model construction. Updated model progress and experimental results will be reported in subsequent versions. All source code, curated datasets, as well as the derived models are disclosed at https://github.com/ADaM-BJTU/O1-CODER . |
| title | o1-Coder: an o1 Replication for Coding |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2412.00154 |