Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2405.10212 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912150296985600 |
|---|---|
| author | Zhao, Jiahao Zhu, Jingwei Tan, Minghuan Yang, Min Li, Renhao Yang, Di Zhang, Chenhao Ye, Guancheng Li, Chengming Hu, Xiping Wong, Derek F. |
| author_facet | Zhao, Jiahao Zhu, Jingwei Tan, Minghuan Yang, Min Li, Renhao Yang, Di Zhang, Chenhao Ye, Guancheng Li, Chengming Hu, Xiping Wong, Derek F. |
| contents | In this paper, we introduce a novel psychological benchmark, CPsyExam, constructed from questions sourced from Chinese language examinations. CPsyExam is designed to prioritize psychological knowledge and case analysis separately, recognizing the significance of applying psychological knowledge to real-world scenarios. From the pool of 22k questions, we utilize 4k to create the benchmark that offers balanced coverage of subjects and incorporates a diverse range of case analysis techniques.Furthermore, we evaluate a range of existing large language models~(LLMs), spanning from open-sourced to API-based models. Our experiments and analysis demonstrate that CPsyExam serves as an effective benchmark for enhancing the understanding of psychology within LLMs and enables the comparison of LLMs across various granularities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_10212 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CPsyExam: A Chinese Benchmark for Evaluating Psychology using Examinations Zhao, Jiahao Zhu, Jingwei Tan, Minghuan Yang, Min Li, Renhao Yang, Di Zhang, Chenhao Ye, Guancheng Li, Chengming Hu, Xiping Wong, Derek F. Computation and Language In this paper, we introduce a novel psychological benchmark, CPsyExam, constructed from questions sourced from Chinese language examinations. CPsyExam is designed to prioritize psychological knowledge and case analysis separately, recognizing the significance of applying psychological knowledge to real-world scenarios. From the pool of 22k questions, we utilize 4k to create the benchmark that offers balanced coverage of subjects and incorporates a diverse range of case analysis techniques.Furthermore, we evaluate a range of existing large language models~(LLMs), spanning from open-sourced to API-based models. Our experiments and analysis demonstrate that CPsyExam serves as an effective benchmark for enhancing the understanding of psychology within LLMs and enables the comparison of LLMs across various granularities. |
| title | CPsyExam: A Chinese Benchmark for Evaluating Psychology using Examinations |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2405.10212 |