Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.16202 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910619082424320 |
|---|---|
| author | Zhang, Qian-Wen Wang, Haochen Li, Fang An, Siyu Qiao, Lingfeng Gao, Liangcai Yin, Di Sun, Xing |
| author_facet | Zhang, Qian-Wen Wang, Haochen Li, Fang An, Siyu Qiao, Lingfeng Gao, Liangcai Yin, Di Sun, Xing |
| contents | Online education platforms have significantly transformed the dissemination of educational resources by providing a dynamic and digital infrastructure. With the further enhancement of this transformation, the advent of Large Language Models (LLMs) has elevated the intelligence levels of these platforms. However, current academic benchmarks provide limited guidance for real-world industry scenarios. This limitation arises because educational applications require more than mere test question responses. To bridge this gap, we introduce CJEval, a benchmark based on Chinese Junior High School Exam Evaluations. CJEval consists of 26,136 samples across four application-level educational tasks covering ten subjects. These samples include not only questions and answers but also detailed annotations such as question types, difficulty levels, knowledge concepts, and answer explanations. By utilizing this benchmark, we assessed LLMs' potential applications and conducted a comprehensive analysis of their performance by fine-tuning on various educational tasks. Extensive experiments and discussions have highlighted the opportunities and challenges of applying LLMs in the field of education. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_16202 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CJEval: A Benchmark for Assessing Large Language Models Using Chinese Junior High School Exam Data Zhang, Qian-Wen Wang, Haochen Li, Fang An, Siyu Qiao, Lingfeng Gao, Liangcai Yin, Di Sun, Xing Artificial Intelligence Online education platforms have significantly transformed the dissemination of educational resources by providing a dynamic and digital infrastructure. With the further enhancement of this transformation, the advent of Large Language Models (LLMs) has elevated the intelligence levels of these platforms. However, current academic benchmarks provide limited guidance for real-world industry scenarios. This limitation arises because educational applications require more than mere test question responses. To bridge this gap, we introduce CJEval, a benchmark based on Chinese Junior High School Exam Evaluations. CJEval consists of 26,136 samples across four application-level educational tasks covering ten subjects. These samples include not only questions and answers but also detailed annotations such as question types, difficulty levels, knowledge concepts, and answer explanations. By utilizing this benchmark, we assessed LLMs' potential applications and conducted a comprehensive analysis of their performance by fine-tuning on various educational tasks. Extensive experiments and discussions have highlighted the opportunities and challenges of applying LLMs in the field of education. |
| title | CJEval: A Benchmark for Assessing Large Language Models Using Chinese Junior High School Exam Data |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2409.16202 |