CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoring
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912383066177536 |
|---|---|
| author | Su, Jiamin Yan, Yibo Gao, Zhuoran Zhang, Han Liu, Xiang Hu, Xuming |
| author_facet | Su, Jiamin Yan, Yibo Gao, Zhuoran Zhang, Han Liu, Xiang Hu, Xuming |
| contents | Automated Essay Scoring (AES) is crucial for modern education, particularly with the increasing prevalence of multimodal assessments. However, traditional AES methods struggle with evaluation generalizability and multimodal perception, while even recent Multimodal Large Language Model (MLLM)-based approaches can produce hallucinated justifications and scores misaligned with human judgment. To address the limitations, we introduce CAFES, the first collaborative multi-agent framework specifically designed for AES. It orchestrates three specialized agents: an Initial Scorer for rapid, trait-specific evaluations; a Feedback Pool Manager to aggregate detailed, evidence-grounded strengths; and a Reflective Scorer that iteratively refines scores based on this feedback to enhance human alignment. Extensive experiments, using state-of-the-art MLLMs, achieve an average relative improvement of 21% in Quadratic Weighted Kappa (QWK) against ground truth, especially for grammatical and lexical diversity. Our proposed CAFES framework paves the way for an intelligent multimodal AES system. The code will be available upon acceptance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_13965 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoring Su, Jiamin Yan, Yibo Gao, Zhuoran Zhang, Han Liu, Xiang Hu, Xuming Computation and Language Artificial Intelligence Automated Essay Scoring (AES) is crucial for modern education, particularly with the increasing prevalence of multimodal assessments. However, traditional AES methods struggle with evaluation generalizability and multimodal perception, while even recent Multimodal Large Language Model (MLLM)-based approaches can produce hallucinated justifications and scores misaligned with human judgment. To address the limitations, we introduce CAFES, the first collaborative multi-agent framework specifically designed for AES. It orchestrates three specialized agents: an Initial Scorer for rapid, trait-specific evaluations; a Feedback Pool Manager to aggregate detailed, evidence-grounded strengths; and a Reflective Scorer that iteratively refines scores based on this feedback to enhance human alignment. Extensive experiments, using state-of-the-art MLLMs, achieve an average relative improvement of 21% in Quadratic Weighted Kappa (QWK) against ground truth, especially for grammatical and lexical diversity. Our proposed CAFES framework paves the way for an intelligent multimodal AES system. The code will be available upon acceptance. |
| title | CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoring |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2505.13965 |