ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910397423943680 |
|---|---|
| author | Xu, Yifan Liu, Xiao Liu, Xinghan Hou, Zhenyu Li, Yueyan Zhang, Xiaohan Wang, Zihan Zeng, Aohan Du, Zhengxiao Zhao, Wenyi Tang, Jie Dong, Yuxiao |
| author_facet | Xu, Yifan Liu, Xiao Liu, Xinghan Hou, Zhenyu Li, Yueyan Zhang, Xiaohan Wang, Zihan Zeng, Aohan Du, Zhengxiao Zhao, Wenyi Tang, Jie Dong, Yuxiao |
| contents | Large language models (LLMs) have shown excellent mastering of human language, but still struggle in real-world applications that require mathematical problem-solving. While many strategies and datasets to enhance LLMs' mathematics are developed, it remains a challenge to simultaneously maintain and improve both language and mathematical capabilities in deployed LLM systems.In this work, we tailor the Self-Critique pipeline, which addresses the challenge in the feedback learning stage of LLM alignment. We first train a general Math-Critique model from the LLM itself to provide feedback signals. Then, we sequentially employ rejective fine-tuning and direct preference optimization over the LLM's own generations for data collection. Based on ChatGLM3-32B, we conduct a series of experiments on both academic and our newly created challenging dataset, MathUserEval. Results show that our pipeline significantly enhances the LLM's mathematical problem-solving while still improving its language ability, outperforming LLMs that could be two times larger. Related techniques have been deployed to ChatGLM\footnote{\url{https://chatglm.cn}}, an online serving LLM. Related evaluation dataset and scripts are released at \url{https://github.com/THUDM/ChatGLM-Math}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_02893 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline Xu, Yifan Liu, Xiao Liu, Xinghan Hou, Zhenyu Li, Yueyan Zhang, Xiaohan Wang, Zihan Zeng, Aohan Du, Zhengxiao Zhao, Wenyi Tang, Jie Dong, Yuxiao Computation and Language Large language models (LLMs) have shown excellent mastering of human language, but still struggle in real-world applications that require mathematical problem-solving. While many strategies and datasets to enhance LLMs' mathematics are developed, it remains a challenge to simultaneously maintain and improve both language and mathematical capabilities in deployed LLM systems.In this work, we tailor the Self-Critique pipeline, which addresses the challenge in the feedback learning stage of LLM alignment. We first train a general Math-Critique model from the LLM itself to provide feedback signals. Then, we sequentially employ rejective fine-tuning and direct preference optimization over the LLM's own generations for data collection. Based on ChatGLM3-32B, we conduct a series of experiments on both academic and our newly created challenging dataset, MathUserEval. Results show that our pipeline significantly enhances the LLM's mathematical problem-solving while still improving its language ability, outperforming LLMs that could be two times larger. Related techniques have been deployed to ChatGLM\footnote{\url{https://chatglm.cn}}, an online serving LLM. Related evaluation dataset and scripts are released at \url{https://github.com/THUDM/ChatGLM-Math}. |
| title | ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2404.02893 |