Chain of Strategy Optimization Makes Large Language Models Better Emotional Supporter
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908546165112832 |
|---|---|
| author | Zhao, Weixiang Sui, Xingyu Han, Xinyang Deng, Yang Hu, Yulin Guo, Jiahe Qin, Libo Du, Qianyun Wang, Shijin Zhao, Yanyan Qin, Bing Liu, Ting |
| author_facet | Zhao, Weixiang Sui, Xingyu Han, Xinyang Deng, Yang Hu, Yulin Guo, Jiahe Qin, Libo Du, Qianyun Wang, Shijin Zhao, Yanyan Qin, Bing Liu, Ting |
| contents | The growing emotional stress in modern society has increased the demand for Emotional Support Conversations (ESC). While Large Language Models (LLMs) show promise for ESC, they face two key challenges: (1) low strategy selection accuracy, and (2) preference bias, limiting their adaptability to emotional needs of users. Existing supervised fine-tuning (SFT) struggles to address these issues, as it rigidly trains models on single gold-standard responses without modeling nuanced strategy trade-offs. To overcome these limitations, we propose Chain-of-Strategy Optimization (CSO), a novel approach that optimizes strategy selection preferences at each dialogue turn. We first leverage Monte Carlo Tree Search to construct ESC-Pro, a high-quality preference dataset with turn-level strategy-response pairs. Training on ESC-Pro with CSO improves both strategy accuracy and bias mitigation, enabling LLMs to generate more empathetic and contextually appropriate responses. Experiments on LLaMA-3.1-8B, Gemma-2-9B, and Qwen2.5-7B demonstrate that CSO outperforms standard SFT, highlighting the efficacy of fine-grained, turn-level preference modeling in ESC. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_05362 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Chain of Strategy Optimization Makes Large Language Models Better Emotional Supporter Zhao, Weixiang Sui, Xingyu Han, Xinyang Deng, Yang Hu, Yulin Guo, Jiahe Qin, Libo Du, Qianyun Wang, Shijin Zhao, Yanyan Qin, Bing Liu, Ting Computation and Language The growing emotional stress in modern society has increased the demand for Emotional Support Conversations (ESC). While Large Language Models (LLMs) show promise for ESC, they face two key challenges: (1) low strategy selection accuracy, and (2) preference bias, limiting their adaptability to emotional needs of users. Existing supervised fine-tuning (SFT) struggles to address these issues, as it rigidly trains models on single gold-standard responses without modeling nuanced strategy trade-offs. To overcome these limitations, we propose Chain-of-Strategy Optimization (CSO), a novel approach that optimizes strategy selection preferences at each dialogue turn. We first leverage Monte Carlo Tree Search to construct ESC-Pro, a high-quality preference dataset with turn-level strategy-response pairs. Training on ESC-Pro with CSO improves both strategy accuracy and bias mitigation, enabling LLMs to generate more empathetic and contextually appropriate responses. Experiments on LLaMA-3.1-8B, Gemma-2-9B, and Qwen2.5-7B demonstrate that CSO outperforms standard SFT, highlighting the efficacy of fine-grained, turn-level preference modeling in ESC. |
| title | Chain of Strategy Optimization Makes Large Language Models Better Emotional Supporter |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.05362 |