Dream to Chat: Model-based Reinforcement Learning on Dialogues with User Belief Modeling
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918148592107520 |
|---|---|
| author | Zhao, Yue Wang, Xiaoyu Wang, Dan Jiang, Zhonglin Gu, Qingqing Chen, Teng Xi, Ningyuan Qu, Jinxian Chen, Yong Ji, Luo |
| author_facet | Zhao, Yue Wang, Xiaoyu Wang, Dan Jiang, Zhonglin Gu, Qingqing Chen, Teng Xi, Ningyuan Qu, Jinxian Chen, Yong Ji, Luo |
| contents | World models have been widely utilized in robotics, gaming, and auto-driving. However, their applications on natural language tasks are relatively limited. In this paper, we construct the dialogue world model, which could predict the user's emotion, sentiment, and intention, and future utterances. By defining a POMDP, we argue emotion, sentiment and intention can be modeled as the user belief and solved by maximizing the information bottleneck. By this user belief modeling, we apply the model-based reinforcement learning framework to the dialogue system, and propose a framework called DreamCUB. Experiments show that the pretrained dialogue world model can achieve state-of-the-art performances on emotion classification and sentiment identification, while dialogue quality is also enhanced by joint training of the policy, critic and dialogue world model. Further analysis shows that this manner holds a reasonable exploration-exploitation balance and also transfers well to out-of-domain scenarios such as empathetic dialogues. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_16876 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dream to Chat: Model-based Reinforcement Learning on Dialogues with User Belief Modeling Zhao, Yue Wang, Xiaoyu Wang, Dan Jiang, Zhonglin Gu, Qingqing Chen, Teng Xi, Ningyuan Qu, Jinxian Chen, Yong Ji, Luo Computation and Language Artificial Intelligence World models have been widely utilized in robotics, gaming, and auto-driving. However, their applications on natural language tasks are relatively limited. In this paper, we construct the dialogue world model, which could predict the user's emotion, sentiment, and intention, and future utterances. By defining a POMDP, we argue emotion, sentiment and intention can be modeled as the user belief and solved by maximizing the information bottleneck. By this user belief modeling, we apply the model-based reinforcement learning framework to the dialogue system, and propose a framework called DreamCUB. Experiments show that the pretrained dialogue world model can achieve state-of-the-art performances on emotion classification and sentiment identification, while dialogue quality is also enhanced by joint training of the policy, critic and dialogue world model. Further analysis shows that this manner holds a reasonable exploration-exploitation balance and also transfers well to out-of-domain scenarios such as empathetic dialogues. |
| title | Dream to Chat: Model-based Reinforcement Learning on Dialogues with User Belief Modeling |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2508.16876 |