RATIONALYST: Mining Implicit Rationales for Process Supervision of Reasoning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908407486742528 |
|---|---|
| author | Jiang, Dongwei Wang, Guoxuan Lu, Yining Wang, Andrew Zhang, Jingyu Liu, Chuyu Van Durme, Benjamin Khashabi, Daniel |
| author_facet | Jiang, Dongwei Wang, Guoxuan Lu, Yining Wang, Andrew Zhang, Jingyu Liu, Chuyu Van Durme, Benjamin Khashabi, Daniel |
| contents | The reasoning steps generated by LLMs might be incomplete, as they mimic logical leaps common in everyday communication found in their pre-training data: underlying rationales are frequently left implicit (unstated). To address this challenge, we introduce RATIONALYST, a model for process-supervision of reasoning based on pre-training on a vast collection of rationale annotations extracted from unlabeled data. We extract 79k rationales from web-scale unlabelled dataset (the Pile) and a combination of reasoning datasets with minimal human intervention. This web-scale pre-training for reasoning allows RATIONALYST to consistently generalize across diverse reasoning tasks, including mathematical, commonsense, scientific, and logical reasoning. Fine-tuned from LLaMa-3-8B, RATIONALYST improves the accuracy of reasoning by an average of 3.9% on 7 representative reasoning benchmarks. It also demonstrates superior performance compared to significantly larger verifiers like GPT-4 and similarly sized models fine-tuned on matching training sets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_01044 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RATIONALYST: Mining Implicit Rationales for Process Supervision of Reasoning Jiang, Dongwei Wang, Guoxuan Lu, Yining Wang, Andrew Zhang, Jingyu Liu, Chuyu Van Durme, Benjamin Khashabi, Daniel Artificial Intelligence Computation and Language The reasoning steps generated by LLMs might be incomplete, as they mimic logical leaps common in everyday communication found in their pre-training data: underlying rationales are frequently left implicit (unstated). To address this challenge, we introduce RATIONALYST, a model for process-supervision of reasoning based on pre-training on a vast collection of rationale annotations extracted from unlabeled data. We extract 79k rationales from web-scale unlabelled dataset (the Pile) and a combination of reasoning datasets with minimal human intervention. This web-scale pre-training for reasoning allows RATIONALYST to consistently generalize across diverse reasoning tasks, including mathematical, commonsense, scientific, and logical reasoning. Fine-tuned from LLaMa-3-8B, RATIONALYST improves the accuracy of reasoning by an average of 3.9% on 7 representative reasoning benchmarks. It also demonstrates superior performance compared to significantly larger verifiers like GPT-4 and similarly sized models fine-tuned on matching training sets. |
| title | RATIONALYST: Mining Implicit Rationales for Process Supervision of Reasoning |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2410.01044 |