Evaluating an LLM-Powered Chatbot for Cognitive Restructuring: Insights from Mental Health Professionals
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929687852220416 |
|---|---|
| author | Wang, Yinzhou Wang, Yimeng Xiao, Ye Escamilla, Liabette Augustine, Bianca Crace, Kelly Zhou, Gang Zhang, Yixuan |
| author_facet | Wang, Yinzhou Wang, Yimeng Xiao, Ye Escamilla, Liabette Augustine, Bianca Crace, Kelly Zhou, Gang Zhang, Yixuan |
| contents | Recent advancements in large language models (LLMs) promise to expand mental health interventions by emulating therapeutic techniques, potentially easing barriers to care. Yet there is a lack of real-world empirical evidence evaluating the strengths and limitations of LLM-enabled psychotherapy interventions. In this work, we evaluate an LLM-powered chatbot, designed via prompt engineering to deliver cognitive restructuring (CR), with 19 users. Mental health professionals then examined the resulting conversation logs to uncover potential benefits and pitfalls. Our findings indicate that an LLM-based CR approach has the capability to adhere to core CR protocols, prompt Socratic questioning, and provide empathetic validation. However, issues of power imbalances, advice-giving, misunderstood cues, and excessive positivity reveal deeper challenges, including the potential to erode therapeutic rapport and ethical concerns. We also discuss design implications for leveraging LLMs in psychotherapy and underscore the importance of expert oversight to mitigate these concerns, which are critical steps toward safer, more effective AI-assisted interventions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_15599 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating an LLM-Powered Chatbot for Cognitive Restructuring: Insights from Mental Health Professionals Wang, Yinzhou Wang, Yimeng Xiao, Ye Escamilla, Liabette Augustine, Bianca Crace, Kelly Zhou, Gang Zhang, Yixuan Human-Computer Interaction Recent advancements in large language models (LLMs) promise to expand mental health interventions by emulating therapeutic techniques, potentially easing barriers to care. Yet there is a lack of real-world empirical evidence evaluating the strengths and limitations of LLM-enabled psychotherapy interventions. In this work, we evaluate an LLM-powered chatbot, designed via prompt engineering to deliver cognitive restructuring (CR), with 19 users. Mental health professionals then examined the resulting conversation logs to uncover potential benefits and pitfalls. Our findings indicate that an LLM-based CR approach has the capability to adhere to core CR protocols, prompt Socratic questioning, and provide empathetic validation. However, issues of power imbalances, advice-giving, misunderstood cues, and excessive positivity reveal deeper challenges, including the potential to erode therapeutic rapport and ethical concerns. We also discuss design implications for leveraging LLMs in psychotherapy and underscore the importance of expert oversight to mitigate these concerns, which are critical steps toward safer, more effective AI-assisted interventions. |
| title | Evaluating an LLM-Powered Chatbot for Cognitive Restructuring: Insights from Mental Health Professionals |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2501.15599 |