Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.15832 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911514268532736 |
|---|---|
| author | Su, Weiwen Zhou, Yuhan Wang, Zihan Yoshinaga, Naoki Toyoda, Masashi |
| author_facet | Su, Weiwen Zhou, Yuhan Wang, Zihan Yoshinaga, Naoki Toyoda, Masashi |
| contents | Existing user simulation approaches focus on generating user-like responses in dialogue. They often assume that the provided persona is sufficient for producing such responses, without verifying whether critical personas are supplied. This raises concerns about the validity of simulation results. To address this issue, we study the task of identifying persona dimensions (e.g., "whether the user is price-sensitive") that are relevant but missing in simulating a user's reply for a given dialogue context. We introduce PICQ-drama (constructed from TVShowGuess), a benchmark of context-aware choice questions, annotated with missing persona dimensions whose absence leads to ambiguous user choices. We further design diverse evaluation criteria for missing persona identification. Benchmarking leading LLMs on our PICQ-drama dataset demonstrates the feasibility of this task. Evaluation across diverse criteria, along with further analyses, reveals cognitive differences between LLMs and humans and highlights the distinct roles of different persona categories in shaping responses. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_15832 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Is He Extroverted? Identifying Missing Relevant Personas for Faithful User Simulation Su, Weiwen Zhou, Yuhan Wang, Zihan Yoshinaga, Naoki Toyoda, Masashi Human-Computer Interaction Artificial Intelligence Existing user simulation approaches focus on generating user-like responses in dialogue. They often assume that the provided persona is sufficient for producing such responses, without verifying whether critical personas are supplied. This raises concerns about the validity of simulation results. To address this issue, we study the task of identifying persona dimensions (e.g., "whether the user is price-sensitive") that are relevant but missing in simulating a user's reply for a given dialogue context. We introduce PICQ-drama (constructed from TVShowGuess), a benchmark of context-aware choice questions, annotated with missing persona dimensions whose absence leads to ambiguous user choices. We further design diverse evaluation criteria for missing persona identification. Benchmarking leading LLMs on our PICQ-drama dataset demonstrates the feasibility of this task. Evaluation across diverse criteria, along with further analyses, reveals cognitive differences between LLMs and humans and highlights the distinct roles of different persona categories in shaping responses. |
| title | Is He Extroverted? Identifying Missing Relevant Personas for Faithful User Simulation |
| topic | Human-Computer Interaction Artificial Intelligence |
| url | https://arxiv.org/abs/2602.15832 |