Rank-Then-Score: Enhancing Large Language Models for Automated Essay Scoring
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908306698665984 |
|---|---|
| author | Cai, Yida Liang, Kun Lee, Sanwoo Wang, Qinghan Wu, Yunfang |
| author_facet | Cai, Yida Liang, Kun Lee, Sanwoo Wang, Qinghan Wu, Yunfang |
| contents | In recent years, large language models (LLMs) achieve remarkable success across a variety of tasks. However, their potential in the domain of Automated Essay Scoring (AES) remains largely underexplored. Moreover, compared to English data, the methods for Chinese AES is not well developed. In this paper, we propose Rank-Then-Score (RTS), a fine-tuning framework based on large language models to enhance their essay scoring capabilities. Specifically, we fine-tune the ranking model (Ranker) with feature-enriched data, and then feed the output of the ranking model, in the form of a candidate score set, with the essay content into the scoring model (Scorer) to produce the final score. Experimental results on two benchmark datasets, HSK and ASAP, demonstrate that RTS consistently outperforms the direct prompting (Vanilla) method in terms of average QWK across all LLMs and datasets, and achieves the best performance on Chinese essay scoring using the HSK dataset. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_05736 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Rank-Then-Score: Enhancing Large Language Models for Automated Essay Scoring Cai, Yida Liang, Kun Lee, Sanwoo Wang, Qinghan Wu, Yunfang Computation and Language Artificial Intelligence In recent years, large language models (LLMs) achieve remarkable success across a variety of tasks. However, their potential in the domain of Automated Essay Scoring (AES) remains largely underexplored. Moreover, compared to English data, the methods for Chinese AES is not well developed. In this paper, we propose Rank-Then-Score (RTS), a fine-tuning framework based on large language models to enhance their essay scoring capabilities. Specifically, we fine-tune the ranking model (Ranker) with feature-enriched data, and then feed the output of the ranking model, in the form of a candidate score set, with the essay content into the scoring model (Scorer) to produce the final score. Experimental results on two benchmark datasets, HSK and ASAP, demonstrate that RTS consistently outperforms the direct prompting (Vanilla) method in terms of average QWK across all LLMs and datasets, and achieves the best performance on Chinese essay scoring using the HSK dataset. |
| title | Rank-Then-Score: Enhancing Large Language Models for Automated Essay Scoring |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2504.05736 |