ERD: A Framework for Improving LLM Reasoning for Cognitive Distortion Classification
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909144796102656 |
|---|---|
| author | Lim, Sehee Kim, Yejin Choi, Chi-Hyun Sohn, Jy-yong Kim, Byung-Hoon |
| author_facet | Lim, Sehee Kim, Yejin Choi, Chi-Hyun Sohn, Jy-yong Kim, Byung-Hoon |
| contents | Improving the accessibility of psychotherapy with the aid of Large Language Models (LLMs) is garnering a significant attention in recent years. Recognizing cognitive distortions from the interviewee's utterances can be an essential part of psychotherapy, especially for cognitive behavioral therapy. In this paper, we propose ERD, which improves LLM-based cognitive distortion classification performance with the aid of additional modules of (1) extracting the parts related to cognitive distortion, and (2) debating the reasoning steps by multiple agents. Our experimental results on a public dataset show that ERD improves the multi-class F1 score as well as binary specificity score. Regarding the latter score, it turns out that our method is effective in debiasing the baseline method which has high false positive rate, especially when the summary of multi-agent debate is provided to LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_14255 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ERD: A Framework for Improving LLM Reasoning for Cognitive Distortion Classification Lim, Sehee Kim, Yejin Choi, Chi-Hyun Sohn, Jy-yong Kim, Byung-Hoon Computation and Language Machine Learning Improving the accessibility of psychotherapy with the aid of Large Language Models (LLMs) is garnering a significant attention in recent years. Recognizing cognitive distortions from the interviewee's utterances can be an essential part of psychotherapy, especially for cognitive behavioral therapy. In this paper, we propose ERD, which improves LLM-based cognitive distortion classification performance with the aid of additional modules of (1) extracting the parts related to cognitive distortion, and (2) debating the reasoning steps by multiple agents. Our experimental results on a public dataset show that ERD improves the multi-class F1 score as well as binary specificity score. Regarding the latter score, it turns out that our method is effective in debiasing the baseline method which has high false positive rate, especially when the summary of multi-agent debate is provided to LLMs. |
| title | ERD: A Framework for Improving LLM Reasoning for Cognitive Distortion Classification |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2403.14255 |