Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.19643 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916864890765312 |
|---|---|
| author | Kim, Minju Yoo, Dongje Hwang, Yeonjun Kang, Minseok Kim, Namyoung Gwak, Minju Kwak, Beong-woo Chae, Hyungjoo Kim, Harim Lee, Yunjoong Kim, Min Hee Jung, Dayi Chung, Kyong-Mee Yeo, Jinyoung |
| author_facet | Kim, Minju Yoo, Dongje Hwang, Yeonjun Kang, Minseok Kim, Namyoung Gwak, Minju Kwak, Beong-woo Chae, Hyungjoo Kim, Harim Lee, Yunjoong Kim, Min Hee Jung, Dayi Chung, Kyong-Mee Yeo, Jinyoung |
| contents | Understanding clients' thoughts and beliefs is fundamental in counseling, yet current evaluations of LLM therapists often fail to assess this ability. Existing evaluation methods rely on client simulators that clearly disclose internal states to the therapist, making it difficult to determine whether an LLM therapist can uncover unexpressed perspectives. To address this limitation, we introduce MindVoyager, a novel evaluation framework featuring a controllable and realistic client simulator which dynamically adapts itself based on the ongoing counseling session, offering a more realistic and challenging evaluation environment. We further introduce evaluation metrics that assess the exploration ability of LLM therapists by measuring their thorough understanding of client's beliefs and thoughts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_19643 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Can You Share Your Story? Modeling Clients' Metacognition and Openness for LLM Therapist Evaluation Kim, Minju Yoo, Dongje Hwang, Yeonjun Kang, Minseok Kim, Namyoung Gwak, Minju Kwak, Beong-woo Chae, Hyungjoo Kim, Harim Lee, Yunjoong Kim, Min Hee Jung, Dayi Chung, Kyong-Mee Yeo, Jinyoung Computers and Society Artificial Intelligence Understanding clients' thoughts and beliefs is fundamental in counseling, yet current evaluations of LLM therapists often fail to assess this ability. Existing evaluation methods rely on client simulators that clearly disclose internal states to the therapist, making it difficult to determine whether an LLM therapist can uncover unexpressed perspectives. To address this limitation, we introduce MindVoyager, a novel evaluation framework featuring a controllable and realistic client simulator which dynamically adapts itself based on the ongoing counseling session, offering a more realistic and challenging evaluation environment. We further introduce evaluation metrics that assess the exploration ability of LLM therapists by measuring their thorough understanding of client's beliefs and thoughts. |
| title | Can You Share Your Story? Modeling Clients' Metacognition and Openness for LLM Therapist Evaluation |
| topic | Computers and Society Artificial Intelligence |
| url | https://arxiv.org/abs/2507.19643 |