Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Recurso digital |
| Lenguaje: | |
| Publicado: |
Zenodo
2025
|
| Acceso en línea: | https://doi.org/10.5281/zenodo.17369216 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
Tabla de Contenidos:
- <h1>CDLM-AA: Cognitive Diagnostic Language Model for Adaptive English Writing Assessment</h1> <h2>Overview</h2> <p><strong>CDLM-AA</strong> is an intelligent <strong>cognitive diagnosis and adaptive assessment system</strong> for English writing ability, integrating <strong>pre-trained language models</strong> (e.g., BERT or GPT) with <strong>cognitive diagnostic theory</strong>. The framework overcomes the limitations of traditional rule-based and human-evaluated writing assessments by providing <strong>scalable</strong>, <strong>context-sensitive</strong>, and <strong>personalized</strong> feedback.</p> <p>At its core, CDLM-AA introduces:</p> <ul> <li><strong>CDLM (Cognitive Diagnostic Language Model)</strong> — a multimodal, transformer-based language architecture aligned with cognitive attributes.</li> <li><strong>Adaptive Assessment Mechanism</strong> — dynamically adjusts task difficulty based on individual learner profiles.</li> <li><strong>Innovative Methodologies</strong> — graphical propagation layers and spatial-temporal attention for fine-grained diagnostic evaluation.</li> </ul> <h2>✨ Features</h2> <h3>1. Cognitive Diagnostic Language Model (CDLM)</h3> <ul> <li>Built on pre-trained language models (e.g., BERT, GPT), fine-tuned on annotated writing samples.</li> <li>Maps writing samples to latent cognitive attributes (grammar, coherence, vocabulary, organization).</li> <li><strong>Multimodal Encoder + Encoder–Decoder</strong> architecture integrates textual and auxiliary inputs.</li> <li><strong>Sparse Mixture-of-Experts</strong> dynamically routes representations for specialized cognitive tasks.<br> <em>See Figure 1 on page 7 for the overall architecture diagram of CDLM.</em></li> </ul> <h3>2. Multimodal Encoder Architecture</h3> <ul> <li>Employs static and dynamic channel mixing for robust feature refinement.</li> <li>Balances adaptability with stable structure to support different writing genres.<br> <em>Figure 2 on page 8 shows the dual-path encoding structure with static/dynamic attention blocks.</em></li> </ul> <h3>3. Graphical Propagation & Attention Mechanisms</h3> <ul> <li><strong>Graphical Propagation Layer</strong> dynamically adjusts weighting of attributes according to learner performance.</li> <li><strong>Spatial-Temporal Attention</strong> focuses on critical textual segments, highlighting grammar, coherence, and structure.</li> <li>Real-time iterative recalibration aligns feedback with learner progress.<br> <em>Figures 3–4 (pages 10–11) illustrate the feature mapping and graph-based attention process.</em></li> </ul> <h3>4. Adaptive Assessment</h3> <ul> <li>Selects tasks using a <strong>utility function</strong> targeting weak attributes.</li> <li>Task difficulty and focus evolve with learner proficiency, ensuring continuous engagement and growth.</li> </ul> <h2> Datasets</h2> <table> <thead> <tr> <th>Dataset</th> <th>Description</th> <th>Purpose</th> </tr> </thead> <tbody> <tr> <td><strong>English Writing Skill Evaluation Dataset</strong></td> <td>Writing samples annotated for grammar, vocabulary, coherence</td> <td>Model training and benchmarking</td> </tr> <tr> <td><strong>Cognitive Writing Assessment Dataset</strong></td> <td>Think-aloud protocols, keystroke logs, cognitive writing behaviors</td> <td>Cognitive process modeling</td> </tr> <tr> <td><strong>Language Model Guided Writing Dataset</strong></td> <td>Human-model interactive writing tasks</td> <td>LLM-assisted feedback testing</td> </tr> <tr> <td><strong>Adaptive English Writing Proficiency Dataset</strong></td> <td>Adaptive tasks adjusted by performance</td> <td>Real-time feedback evaluation</td> </tr> </tbody> </table> <div> <div dir="ltr"><code> </code></div> </div> <h2> Usage</h2> <ul> <li>Attribute mastery scores (grammar, coherence, vocabulary, organization)</li> <li>Attention visualization on critical text segments</li> <li>Personalized writing feedback</li> <li>Adaptive prompt selection for next tasks</li> </ul> <h2> Applications</h2> <ul> <li><strong>Automated, adaptive English writing evaluation</strong></li> <li><strong>Cognitive profiling</strong> for targeted learning interventions</li> <li><strong>Real-time formative feedback</strong> in digital learning environments</li> <li><strong>Scalable assessment</strong> for large educational settings</li> </ul> <h2> Model Components</h2> <table> <thead> <tr> <th>Component</th> <th>Description</th> </tr> </thead> <tbody> <tr> <td><strong>CDLM</strong></td> <td>Transformer-based cognitive diagnostic language model</td> </tr> <tr> <td><strong>Multimodal Encoder</strong></td> <td>Static + dynamic channel mixing for feature integration</td> </tr> <tr> <td><strong>Graphical Propagation Layer</strong></td> <td>Adjusts task focus dynamically</td> </tr> <tr> <td><strong>Spatial-Temporal Attention</strong></td> <td>Highlights key textual elements for feedback</td> </tr> <tr> <td><strong>Adaptive Feedback Loop</strong></td> <td>Iteratively updates learner profile and prompt difficulty</td> </tr> </tbody> </table> <h2> Performance</h2> <table> <thead> <tr> <th>Dataset</th> <th>Accuracy</th> <th>Precision</th> <th>Recall</th> <th>F1 Score</th> </tr> </thead> <tbody> <tr> <td>English Writing Skill Evaluation</td> <td>89.78</td> <td>89.23</td> <td>88.56</td> <td>88.89</td> </tr> <tr> <td>Cognitive Writing Assessment</td> <td>90.67</td> <td>90.12</td> <td>89.56</td> <td>89.89</td> </tr> <tr> <td>Language Model Guided Writing</td> <td>89.12</td> <td>88.56</td> <td>87.90</td> <td>88.23</td> </tr> <tr> <td>Adaptive Writing Proficiency</td> <td>90.03</td> <td>89.60</td> <td>88.94</td> <td>89.27</td> </tr> </tbody> </table> <p> <em>Tables 1–2 on pages 15–16 compare CDLM-AA to ResNet, ViT, I3D, BLIP, DenseNet, and MobileNet, showing consistent gains across all metrics.</em></p> <h2> Future Work</h2> <ul> <li>Expand cross-linguistic support for multilingual writing assessment.</li> <li>Enhance efficiency via lightweight model variants for classroom deployment.</li> <li>Integrate explainable AI for transparent feedback interpretation.</li> <li>Enable <strong>real-time adaptive learning loops</strong> for live educational applications.</li> </ul> <h2> License</h2> <p>This project is licensed under the MIT License.</p> <h2> Acknowledgments</h2> <p>This work was supported by the <strong>Higher Education Research Program of Hainan Province</strong> (Grant No. Hnjg2023-58).<br>Authors: Minghui Chen, Yang Tian, Shenggu Chen, Zhengzheng Huang.<br>This research bridges <strong>cognitive diagnostic theory</strong> with <strong>pre-trained language modeling</strong>, advancing adaptive and scalable English writing assessment.</p>