Dissociation of Faithful and Unfaithful Reasoning in LLMs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913490004869120 |
|---|---|
| author | Yee, Evelyn Li, Alice Tang, Chenyu Jung, Yeon Ho Paturi, Ramamohan Bergen, Leon |
| author_facet | Yee, Evelyn Li, Alice Tang, Chenyu Jung, Yeon Ho Paturi, Ramamohan Bergen, Leon |
| contents | Large language models (LLMs) often improve their performance in downstream tasks when they generate Chain of Thought reasoning text before producing an answer. We investigate how LLMs recover from errors in Chain of Thought. Through analysis of error recovery behaviors, we find evidence for unfaithfulness in Chain of Thought, which occurs when models arrive at the correct answer despite invalid reasoning text. We identify factors that shift LLM recovery behavior: LLMs recover more frequently from obvious errors and in contexts that provide more evidence for the correct answer. Critically, these factors have divergent effects on faithful and unfaithful recoveries. Our results indicate that there are distinct mechanisms driving faithful and unfaithful error recoveries. Selective targeting of these mechanisms may be able to drive down the rate of unfaithful reasoning and improve model interpretability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_15092 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Dissociation of Faithful and Unfaithful Reasoning in LLMs Yee, Evelyn Li, Alice Tang, Chenyu Jung, Yeon Ho Paturi, Ramamohan Bergen, Leon Artificial Intelligence Computation and Language Large language models (LLMs) often improve their performance in downstream tasks when they generate Chain of Thought reasoning text before producing an answer. We investigate how LLMs recover from errors in Chain of Thought. Through analysis of error recovery behaviors, we find evidence for unfaithfulness in Chain of Thought, which occurs when models arrive at the correct answer despite invalid reasoning text. We identify factors that shift LLM recovery behavior: LLMs recover more frequently from obvious errors and in contexts that provide more evidence for the correct answer. Critically, these factors have divergent effects on faithful and unfaithful recoveries. Our results indicate that there are distinct mechanisms driving faithful and unfaithful error recoveries. Selective targeting of these mechanisms may be able to drive down the rate of unfaithful reasoning and improve model interpretability. |
| title | Dissociation of Faithful and Unfaithful Reasoning in LLMs |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2405.15092 |