TinyR1-32B-Preview: Boosting Accuracy with Branch-Merge Distillation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918473255354368 |
|---|---|
| author | Sun, Lin Zhao, Guangxiang Jian, Xiaoqi Wu, Yuhan Lin, Weihong Zhu, Yongfu Shi, Qilong Jia, Change Yuan, Aomufei Tian, Yuxuan Zhang, Linglin Wu, Jinzhu Ran, Junfeng Hu, Sai-er Jiang, Zihan Zhou, Junting Liu, Wenrui Xiao, Xusen Cui, Bin Yang, Tong Zhang, Xiangzheng |
| author_facet | Sun, Lin Zhao, Guangxiang Jian, Xiaoqi Wu, Yuhan Lin, Weihong Zhu, Yongfu Shi, Qilong Jia, Change Yuan, Aomufei Tian, Yuxuan Zhang, Linglin Wu, Jinzhu Ran, Junfeng Hu, Sai-er Jiang, Zihan Zhou, Junting Liu, Wenrui Xiao, Xusen Cui, Bin Yang, Tong Zhang, Xiangzheng |
| contents | The challenge of reducing the size of Large Language Models (LLMs) while maintaining their performance has gained significant attention. However, existing methods, such as model distillation and transfer learning, often fail to achieve high accuracy. To address this limitation, we introduce the Branch-Merge distillation approach, which enhances model compression through two phases: (1) the Branch Phase, where knowledge from a large teacher model is \textit{selectively distilled} into specialized student models via domain-specific supervised fine-tuning (SFT); And (2) the Merge Phase, where these student models are merged to enable cross-domain knowledge transfer and improve generalization. We validate our distillation approach using DeepSeek-R1 as the teacher and DeepSeek-R1-Distill-Qwen-32B as the student. The resulting merged model, TinyR1-32B-Preview, outperforms its counterpart DeepSeek-R1-Distill-Qwen-32B across multiple benchmarks, including Mathematics (+5.5 points), Coding (+4.4 points) and Science (+2.9 points), while achieving near-equal performance to DeepSeek-R1 on AIME 2024. The Branch-Merge distillation approach provides a scalable solution for creating smaller, high-performing LLMs with reduced computational cost and time. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_04872 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TinyR1-32B-Preview: Boosting Accuracy with Branch-Merge Distillation Sun, Lin Zhao, Guangxiang Jian, Xiaoqi Wu, Yuhan Lin, Weihong Zhu, Yongfu Shi, Qilong Jia, Change Yuan, Aomufei Tian, Yuxuan Zhang, Linglin Wu, Jinzhu Ran, Junfeng Hu, Sai-er Jiang, Zihan Zhou, Junting Liu, Wenrui Xiao, Xusen Cui, Bin Yang, Tong Zhang, Xiangzheng Computation and Language Artificial Intelligence The challenge of reducing the size of Large Language Models (LLMs) while maintaining their performance has gained significant attention. However, existing methods, such as model distillation and transfer learning, often fail to achieve high accuracy. To address this limitation, we introduce the Branch-Merge distillation approach, which enhances model compression through two phases: (1) the Branch Phase, where knowledge from a large teacher model is \textit{selectively distilled} into specialized student models via domain-specific supervised fine-tuning (SFT); And (2) the Merge Phase, where these student models are merged to enable cross-domain knowledge transfer and improve generalization. We validate our distillation approach using DeepSeek-R1 as the teacher and DeepSeek-R1-Distill-Qwen-32B as the student. The resulting merged model, TinyR1-32B-Preview, outperforms its counterpart DeepSeek-R1-Distill-Qwen-32B across multiple benchmarks, including Mathematics (+5.5 points), Coding (+4.4 points) and Science (+2.9 points), while achieving near-equal performance to DeepSeek-R1 on AIME 2024. The Branch-Merge distillation approach provides a scalable solution for creating smaller, high-performing LLMs with reduced computational cost and time. |
| title | TinyR1-32B-Preview: Boosting Accuracy with Branch-Merge Distillation |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.04872 |