Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.11463 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909957785387008 |
|---|---|
| author | Lim, Junghwan Lee, Sungmin Kim, Dongseok Kim, Taehyun Park, Eunhwan Lee, Jeesoo Lee, Jeongdoo Lee, Junhyeok Cheung, Wai Ting Choi, Dahye Ha, Minsu Her, Jaeheui Huh, Jaeyeon Jung, Hanbin Kang, Changjin Kim, Beomgyu Kim, Minjae Kim, Taewhan Kim, Youngrok Kweon, Hyukjin Lee, Haesol Lee, Kungyu Oh, Dongpin Park, Yeongjae Ryu, Bokki Weon, Dongjoo |
| author_facet | Lim, Junghwan Lee, Sungmin Kim, Dongseok Kim, Taehyun Park, Eunhwan Lee, Jeesoo Lee, Jeongdoo Lee, Junhyeok Cheung, Wai Ting Choi, Dahye Ha, Minsu Her, Jaeheui Huh, Jaeyeon Jung, Hanbin Kang, Changjin Kim, Beomgyu Kim, Minjae Kim, Taewhan Kim, Youngrok Kweon, Hyukjin Lee, Haesol Lee, Kungyu Oh, Dongpin Park, Yeongjae Ryu, Bokki Weon, Dongjoo |
| contents | We introduce Motif-2-12.7B-Reasoning, a 12.7B parameter language model designed to bridge the gap between open-weight systems and proprietary frontier models in complex reasoning and long-context understanding. Addressing the common challenges of model collapse and training instability in reasoning adaptation, we propose a comprehensive, reproducible training recipe spanning system, data, and algorithmic optimizations. Our approach combines memory-efficient infrastructure for 64K-token contexts using hybrid parallelism and kernel-level optimizations with a two-stage Supervised Fine-Tuning (SFT) curriculum that mitigates distribution mismatch through verified, aligned synthetic data. Furthermore, we detail a robust Reinforcement Learning Fine-Tuning (RLFT) pipeline that stabilizes training via difficulty-aware data filtering and mixed-policy trajectory reuse. Empirical results demonstrate that Motif-2-12.7B-Reasoning achieves performance comparable to models with significantly larger parameter counts across mathematics, coding, and agentic benchmarks, offering the community a competitive open model and a practical blueprint for scaling reasoning capabilities under realistic compute constraints. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_11463 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Motif-2-12.7B-Reasoning: A Practitioner's Guide to RL Training Recipes Lim, Junghwan Lee, Sungmin Kim, Dongseok Kim, Taehyun Park, Eunhwan Lee, Jeesoo Lee, Jeongdoo Lee, Junhyeok Cheung, Wai Ting Choi, Dahye Ha, Minsu Her, Jaeheui Huh, Jaeyeon Jung, Hanbin Kang, Changjin Kim, Beomgyu Kim, Minjae Kim, Taewhan Kim, Youngrok Kweon, Hyukjin Lee, Haesol Lee, Kungyu Oh, Dongpin Park, Yeongjae Ryu, Bokki Weon, Dongjoo Artificial Intelligence We introduce Motif-2-12.7B-Reasoning, a 12.7B parameter language model designed to bridge the gap between open-weight systems and proprietary frontier models in complex reasoning and long-context understanding. Addressing the common challenges of model collapse and training instability in reasoning adaptation, we propose a comprehensive, reproducible training recipe spanning system, data, and algorithmic optimizations. Our approach combines memory-efficient infrastructure for 64K-token contexts using hybrid parallelism and kernel-level optimizations with a two-stage Supervised Fine-Tuning (SFT) curriculum that mitigates distribution mismatch through verified, aligned synthetic data. Furthermore, we detail a robust Reinforcement Learning Fine-Tuning (RLFT) pipeline that stabilizes training via difficulty-aware data filtering and mixed-policy trajectory reuse. Empirical results demonstrate that Motif-2-12.7B-Reasoning achieves performance comparable to models with significantly larger parameter counts across mathematics, coding, and agentic benchmarks, offering the community a competitive open model and a practical blueprint for scaling reasoning capabilities under realistic compute constraints. |
| title | Motif-2-12.7B-Reasoning: A Practitioner's Guide to RL Training Recipes |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2512.11463 |