Large Language Models Cannot Self-Correct Reasoning Yet
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914713646923776 |
|---|---|
| author | Huang, Jie Chen, Xinyun Mishra, Swaroop Zheng, Huaixiu Steven Yu, Adams Wei Song, Xinying Zhou, Denny |
| author_facet | Huang, Jie Chen, Xinyun Mishra, Swaroop Zheng, Huaixiu Steven Yu, Adams Wei Song, Xinying Zhou, Denny |
| contents | Large Language Models (LLMs) have emerged as a groundbreaking technology with their unparalleled text generation capabilities across various applications. Nevertheless, concerns persist regarding the accuracy and appropriateness of their generated content. A contemporary methodology, self-correction, has been proposed as a remedy to these issues. Building upon this premise, this paper critically examines the role and efficacy of self-correction within LLMs, shedding light on its true potential and limitations. Central to our investigation is the notion of intrinsic self-correction, whereby an LLM attempts to correct its initial responses based solely on its inherent capabilities, without the crutch of external feedback. In the context of reasoning, our research indicates that LLMs struggle to self-correct their responses without external feedback, and at times, their performance even degrades after self-correction. Drawing from these insights, we offer suggestions for future research and practical applications in this field. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_01798 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Large Language Models Cannot Self-Correct Reasoning Yet Huang, Jie Chen, Xinyun Mishra, Swaroop Zheng, Huaixiu Steven Yu, Adams Wei Song, Xinying Zhou, Denny Computation and Language Artificial Intelligence Large Language Models (LLMs) have emerged as a groundbreaking technology with their unparalleled text generation capabilities across various applications. Nevertheless, concerns persist regarding the accuracy and appropriateness of their generated content. A contemporary methodology, self-correction, has been proposed as a remedy to these issues. Building upon this premise, this paper critically examines the role and efficacy of self-correction within LLMs, shedding light on its true potential and limitations. Central to our investigation is the notion of intrinsic self-correction, whereby an LLM attempts to correct its initial responses based solely on its inherent capabilities, without the crutch of external feedback. In the context of reasoning, our research indicates that LLMs struggle to self-correct their responses without external feedback, and at times, their performance even degrades after self-correction. Drawing from these insights, we offer suggestions for future research and practical applications in this field. |
| title | Large Language Models Cannot Self-Correct Reasoning Yet |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2310.01798 |