CPsyExam: A Chinese Benchmark for Evaluating Psychology using Examinations
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912150296985600 |
|---|---|
| author | Zhao, Jiahao Zhu, Jingwei Tan, Minghuan Yang, Min Li, Renhao Yang, Di Zhang, Chenhao Ye, Guancheng Li, Chengming Hu, Xiping Wong, Derek F. |
| author_facet | Zhao, Jiahao Zhu, Jingwei Tan, Minghuan Yang, Min Li, Renhao Yang, Di Zhang, Chenhao Ye, Guancheng Li, Chengming Hu, Xiping Wong, Derek F. |
| contents | In this paper, we introduce a novel psychological benchmark, CPsyExam, constructed from questions sourced from Chinese language examinations. CPsyExam is designed to prioritize psychological knowledge and case analysis separately, recognizing the significance of applying psychological knowledge to real-world scenarios. From the pool of 22k questions, we utilize 4k to create the benchmark that offers balanced coverage of subjects and incorporates a diverse range of case analysis techniques.Furthermore, we evaluate a range of existing large language models~(LLMs), spanning from open-sourced to API-based models. Our experiments and analysis demonstrate that CPsyExam serves as an effective benchmark for enhancing the understanding of psychology within LLMs and enables the comparison of LLMs across various granularities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_10212 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CPsyExam: A Chinese Benchmark for Evaluating Psychology using Examinations Zhao, Jiahao Zhu, Jingwei Tan, Minghuan Yang, Min Li, Renhao Yang, Di Zhang, Chenhao Ye, Guancheng Li, Chengming Hu, Xiping Wong, Derek F. Computation and Language In this paper, we introduce a novel psychological benchmark, CPsyExam, constructed from questions sourced from Chinese language examinations. CPsyExam is designed to prioritize psychological knowledge and case analysis separately, recognizing the significance of applying psychological knowledge to real-world scenarios. From the pool of 22k questions, we utilize 4k to create the benchmark that offers balanced coverage of subjects and incorporates a diverse range of case analysis techniques.Furthermore, we evaluate a range of existing large language models~(LLMs), spanning from open-sourced to API-based models. Our experiments and analysis demonstrate that CPsyExam serves as an effective benchmark for enhancing the understanding of psychology within LLMs and enables the comparison of LLMs across various granularities. |
| title | CPsyExam: A Chinese Benchmark for Evaluating Psychology using Examinations |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2405.10212 |