WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910984284667904 |
|---|---|
| author | Luo, Haipeng Sun, Qingfeng Xu, Can Zhao, Pu Lou, Jianguang Tao, Chongyang Geng, Xiubo Lin, Qingwei Chen, Shifeng Tang, Yansong Zhang, Dongmei |
| author_facet | Luo, Haipeng Sun, Qingfeng Xu, Can Zhao, Pu Lou, Jianguang Tao, Chongyang Geng, Xiubo Lin, Qingwei Chen, Shifeng Tang, Yansong Zhang, Dongmei |
| contents | Large language models (LLMs), such as GPT-4, have shown remarkable performance in natural language processing (NLP) tasks, including challenging mathematical reasoning. However, most existing open-source models are only pre-trained on large-scale internet data and without math-related optimization. In this paper, we present WizardMath, which enhances the mathematical CoT reasoning abilities of LLMs without using external python tools, by applying our proposed Reinforcement Learning from Evol-Instruct Feedback (RLEIF) method to the domain of math. Through extensive experiments on two mathematical reasoning benchmarks, namely GSM8k and MATH, we reveal the extraordinary capabilities of our model. Remarkably, WizardMath-Mistral 7B surpasses top-tier open-source LLMs by a substantial margin with higher data efficiency. Furthermore, WizardMath 70B even outperforms GPT-3.5-Turbo, Claude 2, Gemini Pro and GPT-4-early-version. Additionally, our preliminary exploration highlights the pivotal role of instruction evolution and process supervision in achieving exceptional math performance. For more details refer to https://github.com/nlpxucan/WizardLM |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_09583 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct Luo, Haipeng Sun, Qingfeng Xu, Can Zhao, Pu Lou, Jianguang Tao, Chongyang Geng, Xiubo Lin, Qingwei Chen, Shifeng Tang, Yansong Zhang, Dongmei Computation and Language Artificial Intelligence Machine Learning Large language models (LLMs), such as GPT-4, have shown remarkable performance in natural language processing (NLP) tasks, including challenging mathematical reasoning. However, most existing open-source models are only pre-trained on large-scale internet data and without math-related optimization. In this paper, we present WizardMath, which enhances the mathematical CoT reasoning abilities of LLMs without using external python tools, by applying our proposed Reinforcement Learning from Evol-Instruct Feedback (RLEIF) method to the domain of math. Through extensive experiments on two mathematical reasoning benchmarks, namely GSM8k and MATH, we reveal the extraordinary capabilities of our model. Remarkably, WizardMath-Mistral 7B surpasses top-tier open-source LLMs by a substantial margin with higher data efficiency. Furthermore, WizardMath 70B even outperforms GPT-3.5-Turbo, Claude 2, Gemini Pro and GPT-4-early-version. Additionally, our preliminary exploration highlights the pivotal role of instruction evolution and process supervision in achieving exceptional math performance. For more details refer to https://github.com/nlpxucan/WizardLM |
| title | WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2308.09583 |