Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911793213865984 |
|---|---|
| author | Zheng, Huaixiu Steven Mishra, Swaroop Chen, Xinyun Cheng, Heng-Tze Chi, Ed H. Le, Quoc V Zhou, Denny |
| author_facet | Zheng, Huaixiu Steven Mishra, Swaroop Chen, Xinyun Cheng, Heng-Tze Chi, Ed H. Le, Quoc V Zhou, Denny |
| contents | We present Step-Back Prompting, a simple prompting technique that enables LLMs to do abstractions to derive high-level concepts and first principles from instances containing specific details. Using the concepts and principles to guide reasoning, LLMs significantly improve their abilities in following a correct reasoning path towards the solution. We conduct experiments of Step-Back Prompting with PaLM-2L, GPT-4 and Llama2-70B models, and observe substantial performance gains on various challenging reasoning-intensive tasks including STEM, Knowledge QA, and Multi-Hop Reasoning. For instance, Step-Back Prompting improves PaLM-2L performance on MMLU (Physics and Chemistry) by 7% and 11% respectively, TimeQA by 27%, and MuSiQue by 7%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_06117 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models Zheng, Huaixiu Steven Mishra, Swaroop Chen, Xinyun Cheng, Heng-Tze Chi, Ed H. Le, Quoc V Zhou, Denny Machine Learning Artificial Intelligence Computation and Language We present Step-Back Prompting, a simple prompting technique that enables LLMs to do abstractions to derive high-level concepts and first principles from instances containing specific details. Using the concepts and principles to guide reasoning, LLMs significantly improve their abilities in following a correct reasoning path towards the solution. We conduct experiments of Step-Back Prompting with PaLM-2L, GPT-4 and Llama2-70B models, and observe substantial performance gains on various challenging reasoning-intensive tasks including STEM, Knowledge QA, and Multi-Hop Reasoning. For instance, Step-Back Prompting improves PaLM-2L performance on MMLU (Physics and Chemistry) by 7% and 11% respectively, TimeQA by 27%, and MuSiQue by 7%. |
| title | Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models |
| topic | Machine Learning Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2310.06117 |