Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.05129 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911203458023424 |
|---|---|
| author | Xu, Qingshu Jiao, Hong Zhou, Tianyi Li, Ming Zhang, Nan Peters, Sydney Fu, Yanbin |
| author_facet | Xu, Qingshu Jiao, Hong Zhou, Tianyi Li, Ming Zhang, Nan Peters, Sydney Fu, Yanbin |
| contents | Accurate alignment of items to content standards is critical for valid score interpretation in large-scale assessments. This study evaluates three automated paradigms for aligning items with four domain and nineteen skill labels. First, we extracted embeddings and trained multiple classical supervised machine learning models, and further investigated the impact of dimensionality reduction on model performance. Second, we fine-tuned eight BERT model and its variants for both domain and skill alignment. Third, we explored ensemble learning with majority voting and stacking with multiple meta-models. The DeBERTa-v3-base achieved the highest weighted-average F1 score of 0.950 for domain alignment while the RoBERTa-large yielded the highest F1 score of 0.869 for skill alignment. Ensemble models did not surpass the best-performing language models. Dimension reduction enhanced linear classifiers based on embeddings but did not perform better than language models. This study demonstrated different methods in automated item alignment to content standards.} |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_05129 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Automated Alignment of Math Items to Content Standards in Large-Scale Assessments Using Language Models Xu, Qingshu Jiao, Hong Zhou, Tianyi Li, Ming Zhang, Nan Peters, Sydney Fu, Yanbin Computation and Language Machine Learning Accurate alignment of items to content standards is critical for valid score interpretation in large-scale assessments. This study evaluates three automated paradigms for aligning items with four domain and nineteen skill labels. First, we extracted embeddings and trained multiple classical supervised machine learning models, and further investigated the impact of dimensionality reduction on model performance. Second, we fine-tuned eight BERT model and its variants for both domain and skill alignment. Third, we explored ensemble learning with majority voting and stacking with multiple meta-models. The DeBERTa-v3-base achieved the highest weighted-average F1 score of 0.950 for domain alignment while the RoBERTa-large yielded the highest F1 score of 0.869 for skill alignment. Ensemble models did not surpass the best-performing language models. Dimension reduction enhanced linear classifiers based on embeddings but did not perform better than language models. This study demonstrated different methods in automated item alignment to content standards.} |
| title | Automated Alignment of Math Items to Content Standards in Large-Scale Assessments Using Language Models |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2510.05129 |