Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.12326 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914928609198080 |
|---|---|
| author | Yang, Qisen Wang, Zekun Chen, Honghui Wang, Shenzhi Pu, Yifan Gao, Xin Huang, Wenhao Song, Shiji Huang, Gao |
| author_facet | Yang, Qisen Wang, Zekun Chen, Honghui Wang, Shenzhi Pu, Yifan Gao, Xin Huang, Wenhao Song, Shiji Huang, Gao |
| contents | Psychological measurement is essential for mental health, self-understanding, and personal development. Traditional methods, such as self-report scales and psychologist interviews, often face challenges with engagement and accessibility. While game-based and LLM-based tools have been explored to improve user interest and automate assessment, they struggle to balance engagement with generalizability. In this work, we propose PsychoGAT (Psychological Game AgenTs) to achieve a generic gamification of psychological assessment. The main insight is that powerful LLMs can function both as adept psychologists and innovative game designers. By incorporating LLM agents into designated roles and carefully managing their interactions, PsychoGAT can transform any standardized scales into personalized and engaging interactive fiction games. To validate the proposed method, we conduct psychometric evaluations to assess its effectiveness and employ human evaluators to examine the generated content across various psychological constructs, including depression, cognitive distortions, and personality traits. Results demonstrate that PsychoGAT serves as an effective assessment tool, achieving statistically significant excellence in psychometric metrics such as reliability, convergent validity, and discriminant validity. Moreover, human evaluations confirm PsychoGAT's enhancements in content coherence, interactivity, interest, immersion, and satisfaction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_12326 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agents Yang, Qisen Wang, Zekun Chen, Honghui Wang, Shenzhi Pu, Yifan Gao, Xin Huang, Wenhao Song, Shiji Huang, Gao Computation and Language Computers and Society Human-Computer Interaction Machine Learning Multiagent Systems Psychological measurement is essential for mental health, self-understanding, and personal development. Traditional methods, such as self-report scales and psychologist interviews, often face challenges with engagement and accessibility. While game-based and LLM-based tools have been explored to improve user interest and automate assessment, they struggle to balance engagement with generalizability. In this work, we propose PsychoGAT (Psychological Game AgenTs) to achieve a generic gamification of psychological assessment. The main insight is that powerful LLMs can function both as adept psychologists and innovative game designers. By incorporating LLM agents into designated roles and carefully managing their interactions, PsychoGAT can transform any standardized scales into personalized and engaging interactive fiction games. To validate the proposed method, we conduct psychometric evaluations to assess its effectiveness and employ human evaluators to examine the generated content across various psychological constructs, including depression, cognitive distortions, and personality traits. Results demonstrate that PsychoGAT serves as an effective assessment tool, achieving statistically significant excellence in psychometric metrics such as reliability, convergent validity, and discriminant validity. Moreover, human evaluations confirm PsychoGAT's enhancements in content coherence, interactivity, interest, immersion, and satisfaction. |
| title | PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agents |
| topic | Computation and Language Computers and Society Human-Computer Interaction Machine Learning Multiagent Systems |
| url | https://arxiv.org/abs/2402.12326 |