Boosting Deductive Reasoning with Step Signals In RLHF
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917813531181056 |
|---|---|
| author | Li, Jialian Zhang, Yipin Shen, Wei Yan, Yuzi Xie, Jian Yan, Dong |
| author_facet | Li, Jialian Zhang, Yipin Shen, Wei Yan, Yuzi Xie, Jian Yan, Dong |
| contents | Logical reasoning is a crucial task for Large Language Models (LLMs), enabling them to tackle complex problems. Among reasoning tasks, multi-step reasoning poses a particular challenge. Grounded in the theory of formal logic, we have developed an automated method, Multi-step Deduction (MuseD), for deductive reasoning data. MuseD has allowed us to create training and testing datasets for multi-step reasoning. Our generation method enables control over the complexity of the generated instructions, facilitating training and evaluation of models across different difficulty levels. Through RLHF training, our training data has demonstrated significant improvements in logical capabilities for both in-domain of out-of-domain reasoning tasks. Additionally, we have conducted tests to assess the multi-step reasoning abilities of various models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_09528 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Boosting Deductive Reasoning with Step Signals In RLHF Li, Jialian Zhang, Yipin Shen, Wei Yan, Yuzi Xie, Jian Yan, Dong Machine Learning Artificial Intelligence Logical reasoning is a crucial task for Large Language Models (LLMs), enabling them to tackle complex problems. Among reasoning tasks, multi-step reasoning poses a particular challenge. Grounded in the theory of formal logic, we have developed an automated method, Multi-step Deduction (MuseD), for deductive reasoning data. MuseD has allowed us to create training and testing datasets for multi-step reasoning. Our generation method enables control over the complexity of the generated instructions, facilitating training and evaluation of models across different difficulty levels. Through RLHF training, our training data has demonstrated significant improvements in logical capabilities for both in-domain of out-of-domain reasoning tasks. Additionally, we have conducted tests to assess the multi-step reasoning abilities of various models. |
| title | Boosting Deductive Reasoning with Step Signals In RLHF |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2410.09528 |