Focus on Your Question! Interpreting and Mitigating Toxic CoT Problems in Commonsense Reasoning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916434298273792 |
|---|---|
| author | Li, Jiachun Cao, Pengfei Wang, Chenhao Jin, Zhuoran Chen, Yubo Zeng, Daojian Liu, Kang Zhao, Jun |
| author_facet | Li, Jiachun Cao, Pengfei Wang, Chenhao Jin, Zhuoran Chen, Yubo Zeng, Daojian Liu, Kang Zhao, Jun |
| contents | Large language models exhibit high-level commonsense reasoning abilities, especially with enhancement methods like Chain-of-Thought (CoT). However, we find these CoT-like methods lead to a considerable number of originally correct answers turning wrong, which we define as the Toxic CoT problem. To interpret and mitigate this problem, we first utilize attribution tracing and causal tracing methods to probe the internal working mechanism of the LLM during CoT reasoning. Through comparisons, we prove that the model exhibits information loss from the question over the shallow attention layers when generating rationales or answers. Based on the probing findings, we design a novel method called RIDERS (Residual decodIng and sERial-position Swap), which compensates for the information deficit in the model from both decoding and serial-position perspectives. Through extensive experiments on multiple commonsense reasoning benchmarks, we validate that this method not only significantly eliminates Toxic CoT problems (decreased by 23.6%), but also effectively improves the model's overall commonsense reasoning performance (increased by 5.5%). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_18344 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Focus on Your Question! Interpreting and Mitigating Toxic CoT Problems in Commonsense Reasoning Li, Jiachun Cao, Pengfei Wang, Chenhao Jin, Zhuoran Chen, Yubo Zeng, Daojian Liu, Kang Zhao, Jun Computation and Language Artificial Intelligence Large language models exhibit high-level commonsense reasoning abilities, especially with enhancement methods like Chain-of-Thought (CoT). However, we find these CoT-like methods lead to a considerable number of originally correct answers turning wrong, which we define as the Toxic CoT problem. To interpret and mitigate this problem, we first utilize attribution tracing and causal tracing methods to probe the internal working mechanism of the LLM during CoT reasoning. Through comparisons, we prove that the model exhibits information loss from the question over the shallow attention layers when generating rationales or answers. Based on the probing findings, we design a novel method called RIDERS (Residual decodIng and sERial-position Swap), which compensates for the information deficit in the model from both decoding and serial-position perspectives. Through extensive experiments on multiple commonsense reasoning benchmarks, we validate that this method not only significantly eliminates Toxic CoT problems (decreased by 23.6%), but also effectively improves the model's overall commonsense reasoning performance (increased by 5.5%). |
| title | Focus on Your Question! Interpreting and Mitigating Toxic CoT Problems in Commonsense Reasoning |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2402.18344 |