RecUserSim: A Realistic and Diverse User Simulator for Evaluating Conversational Recommender Systems
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912511382519808 |
|---|---|
| author | Chen, Luyu Dai, Quanyu Zhang, Zeyu Feng, Xueyang Zhang, Mingyu Tang, Pengcheng Chen, Xu Zhu, Yue Dong, Zhenhua |
| author_facet | Chen, Luyu Dai, Quanyu Zhang, Zeyu Feng, Xueyang Zhang, Mingyu Tang, Pengcheng Chen, Xu Zhu, Yue Dong, Zhenhua |
| contents | Conversational recommender systems (CRS) enhance user experience through multi-turn interactions, yet evaluating CRS remains challenging. User simulators can provide comprehensive evaluations through interactions with CRS, but building realistic and diverse simulators is difficult. While recent work leverages large language models (LLMs) to simulate user interactions, they still fall short in emulating individual real users across diverse scenarios and lack explicit rating mechanisms for quantitative evaluation. To address these gaps, we propose RecUserSim, an LLM agent-based user simulator with enhanced simulation realism and diversity while providing explicit scores. RecUserSim features several key modules: a profile module for defining realistic and diverse user personas, a memory module for tracking interaction history and discovering unknown preferences, and a core action module inspired by Bounded Rationality theory that enables nuanced decision-making while generating more fine-grained actions and personalized responses. To further enhance output control, a refinement module is designed to fine-tune final responses. Experiments demonstrate that RecUserSim generates diverse, controllable outputs and produces realistic, high-quality dialogues, even with smaller base LLMs. The ratings generated by RecUserSim show high consistency across different base LLMs, highlighting its effectiveness for CRS evaluation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_22897 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RecUserSim: A Realistic and Diverse User Simulator for Evaluating Conversational Recommender Systems Chen, Luyu Dai, Quanyu Zhang, Zeyu Feng, Xueyang Zhang, Mingyu Tang, Pengcheng Chen, Xu Zhu, Yue Dong, Zhenhua Human-Computer Interaction Artificial Intelligence Conversational recommender systems (CRS) enhance user experience through multi-turn interactions, yet evaluating CRS remains challenging. User simulators can provide comprehensive evaluations through interactions with CRS, but building realistic and diverse simulators is difficult. While recent work leverages large language models (LLMs) to simulate user interactions, they still fall short in emulating individual real users across diverse scenarios and lack explicit rating mechanisms for quantitative evaluation. To address these gaps, we propose RecUserSim, an LLM agent-based user simulator with enhanced simulation realism and diversity while providing explicit scores. RecUserSim features several key modules: a profile module for defining realistic and diverse user personas, a memory module for tracking interaction history and discovering unknown preferences, and a core action module inspired by Bounded Rationality theory that enables nuanced decision-making while generating more fine-grained actions and personalized responses. To further enhance output control, a refinement module is designed to fine-tune final responses. Experiments demonstrate that RecUserSim generates diverse, controllable outputs and produces realistic, high-quality dialogues, even with smaller base LLMs. The ratings generated by RecUserSim show high consistency across different base LLMs, highlighting its effectiveness for CRS evaluation. |
| title | RecUserSim: A Realistic and Diverse User Simulator for Evaluating Conversational Recommender Systems |
| topic | Human-Computer Interaction Artificial Intelligence |
| url | https://arxiv.org/abs/2507.22897 |