Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.11678 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914179116433408 |
|---|---|
| author | Li, Haoxuan Yu, Jifan Cong, Xin Dang, Yang Zhang-li, Daniel Mi, Lu Zhan, Yisi Liu, Huiqin Liu, Zhiyuan |
| author_facet | Li, Haoxuan Yu, Jifan Cong, Xin Dang, Yang Zhang-li, Daniel Mi, Lu Zhan, Yisi Liu, Huiqin Liu, Zhiyuan |
| contents | While rapid advances in large language models (LLMs) are reshaping data-driven intelligent education, accurately simulating students remains an important but challenging bottleneck for scalable educational data collection, evaluation, and intervention design. However, current works are limited by scarce real interaction data, costly expert evaluation for realism, and a lack of large-scale, systematic analyses of LLMs ability in simulating students. We address this gap by presenting a three-stage LLM-human collaborative pipeline to automatically generate and filter high-quality student agents. We leverage a two-round automated scoring validated by human experts and deploy a score propagation module to obtain more consistent scores across the student similarity graph. Experiments show that combining automated scoring, expert calibration, and graph-based propagation yields simulated student that more closely track authentication by human judgments. We then analyze which profiles and behaviors are simulated more faithfully, supporting subsequent studies on personalized learning and educational assessment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_11678 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Which Type of Students can LLMs Act? Investigating Authentic Simulation with Graph-based Human-AI Collaborative System Li, Haoxuan Yu, Jifan Cong, Xin Dang, Yang Zhang-li, Daniel Mi, Lu Zhan, Yisi Liu, Huiqin Liu, Zhiyuan Computers and Society Computation and Language While rapid advances in large language models (LLMs) are reshaping data-driven intelligent education, accurately simulating students remains an important but challenging bottleneck for scalable educational data collection, evaluation, and intervention design. However, current works are limited by scarce real interaction data, costly expert evaluation for realism, and a lack of large-scale, systematic analyses of LLMs ability in simulating students. We address this gap by presenting a three-stage LLM-human collaborative pipeline to automatically generate and filter high-quality student agents. We leverage a two-round automated scoring validated by human experts and deploy a score propagation module to obtain more consistent scores across the student similarity graph. Experiments show that combining automated scoring, expert calibration, and graph-based propagation yields simulated student that more closely track authentication by human judgments. We then analyze which profiles and behaviors are simulated more faithfully, supporting subsequent studies on personalized learning and educational assessment. |
| title | Which Type of Students can LLMs Act? Investigating Authentic Simulation with Graph-based Human-AI Collaborative System |
| topic | Computers and Society Computation and Language |
| url | https://arxiv.org/abs/2502.11678 |