Guardado en:
| Autores principales: | , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2406.14024 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914977494859776 |
|---|---|
| author | Gao, Bofei Cai, Zefan Xu, Runxin Wang, Peiyi Zheng, Ce Lin, Runji Lu, Keming Liu, Dayiheng Zhou, Chang Xiao, Wen Hu, Junjie Liu, Tianyu Chang, Baobao |
| author_facet | Gao, Bofei Cai, Zefan Xu, Runxin Wang, Peiyi Zheng, Ce Lin, Runji Lu, Keming Liu, Dayiheng Zhou, Chang Xiao, Wen Hu, Junjie Liu, Tianyu Chang, Baobao |
| contents | In recent progress, mathematical verifiers have achieved success in mathematical reasoning tasks by validating the correctness of solutions generated by policy models. However, existing verifiers are trained with binary classification labels, which are not informative enough for the model to accurately assess the solutions. To mitigate the aforementioned insufficiency of binary labels, we introduce step-wise natural language feedback as rationale labels, that is, the correctness of each step and the detailed explanations. In this paper, we propose Math-Minos, a natural language feedback-enhanced verifier by constructing automatically generated training data and a two-stage training paradigm for effective training and efficient inference. Our experiments reveal that a small set of natural language feedback can significantly boost the performance of the verifier in both verification and reinforcement learning. We have released the code and data for further exploration. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_14024 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback Gao, Bofei Cai, Zefan Xu, Runxin Wang, Peiyi Zheng, Ce Lin, Runji Lu, Keming Liu, Dayiheng Zhou, Chang Xiao, Wen Hu, Junjie Liu, Tianyu Chang, Baobao Computation and Language In recent progress, mathematical verifiers have achieved success in mathematical reasoning tasks by validating the correctness of solutions generated by policy models. However, existing verifiers are trained with binary classification labels, which are not informative enough for the model to accurately assess the solutions. To mitigate the aforementioned insufficiency of binary labels, we introduce step-wise natural language feedback as rationale labels, that is, the correctness of each step and the detailed explanations. In this paper, we propose Math-Minos, a natural language feedback-enhanced verifier by constructing automatically generated training data and a two-stage training paradigm for effective training and efficient inference. Our experiments reveal that a small set of natural language feedback can significantly boost the performance of the verifier in both verification and reinforcement learning. We have released the code and data for further exploration. |
| title | LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2406.14024 |