Cover Image

Beyond Agreement: Rethinking Ground Truth in Educational AI Annotation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Thomas, Danielle R., Borchers, Conrad, Koedinger, Kenneth R.
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Computers and Society
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Modernizing Ground Truth: Four Shifts Toward Improving Reliability and Validity in AI in Education
by: Thomas, Danielle R., et al.
Published: (2026)

Augmenting Human-Annotated Training Data with Large Language Model Generation and Distillation in Open-Response Assessment
by: Borchers, Conrad, et al.
Published: (2025)

Improving Assessment of Tutoring Practices using Retrieval-Augmented Generation
by: Han, Zifei FeiFei, et al.
Published: (2024)

Using Large Language Models to Assess Tutors' Performance in Reacting to Students Making Math Errors
by: Kakarla, Sanjit, et al.
Published: (2024)

Situated Ground Truths: Enhancing Bias-Aware AI by Situating Data Labels with SituAnnotate
by: Pandiani, Delfina Sol Martinez, et al.
Published: (2024)

Brief but Impactful: How Human Tutoring Interactions Shape Engagement in Online Learning
by: Borchers, Conrad, et al.
Published: (2026)

Do Tutors Learn from Equity Training and Can Generative AI Assess It?
by: Thomas, Danielle R., et al.
Published: (2024)

LLM-Generated Feedback Supports Learning If Learners Choose to Use It
by: Thomas, Danielle R., et al.
Published: (2025)

Does Multiple Choice Have a Future in the Age of Generative AI? A Posttest-only RCT
by: Thomas, Danielle R., et al.
Published: (2024)

An Integrated Platform for Studying Learning with Intelligent Tutoring Systems: CTAT+TutorShop
by: Aleven, Vincent, et al.
Published: (2025)

The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation
by: Munir, Sheza, et al.
Published: (2026)

How to Assess AI Literacy: Misalignment Between Self-Reported and Objective-Based Measures
by: Zhang, Shan, et al.
Published: (2026)

3DG: A Framework for Using Generative AI for Handling Sparse Learner Performance Data From Intelligent Tutoring Systems
by: Zhang, Liang, et al.
Published: (2024)

Truth Knows No Language: Evaluating Truthfulness Beyond English
by: Figueras, Blanca Calvo, et al.
Published: (2025)

Optimizing Mastery Learning by Fast-Forwarding Over-Practice Steps
by: Xia, Meng, et al.
Published: (2025)

Beyond Automation: Rethinking Work, Creativity, and Governance in the Age of Generative AI
by: Lin, Haocheng
Published: (2025)

Synthetic Data and the Shifting Ground of Truth
by: Offenhuber, Dietmar
Published: (2025)

Combining Large Language Models with Tutoring System Intelligence: A Case Study in Caregiver Homework Support
by: Venugopalan, Devika, et al.
Published: (2024)

Rethinking AI Literacy Education in Higher Education: Bridging Risk Perception and Responsible Adoption
by: Yu, Shasha, et al.
Published: (2026)

Leveraging LLMs to Assess Tutor Moves in Real-Life Dialogues: A Feasibility Study
by: Thomas, Danielle R., et al.
Published: (2025)

Whose Truth? Pluralistic Geo-Alignment for (Agentic) AI
by: Janowicz, Krzysztof, et al.
Published: (2025)

Evaluating a Data-Driven Redesign Process for Intelligent Tutoring Systems
by: Lyu, Qianru, et al.
Published: (2026)

Fairness Evaluation for Uplift Modeling in the Absence of Ground Truth
by: Kadioglu, Serdar, et al.
Published: (2024)

Evidence of a Cognitive Shift in AI Education: How Students Are Rethinking Human Intelligence?
by: Rekik, Islem
Published: (2026)

International Agreements on AI Safety: Review and Recommendations for a Conditional AI Safety Treaty
by: Scholefield, Rebecca, et al.
Published: (2025)

Benchmarking Educational LLMs with Analytics: A Case Study on Gender Bias in Feedback
by: Du, Yishan, et al.
Published: (2025)

Escaping the Agreement Trap: Defensibility Signals for Evaluating Rule-Governed AI
by: O'Herlihy, Michael, et al.
Published: (2026)

Rethinking AI Cultural Alignment
by: Bravansky, Michal, et al.
Published: (2025)

Starting Seatwork Earlier as a Valid Measure of Student Engagement
by: Gurung, Ashish, et al.
Published: (2025)

TruthStance: An Annotated Dataset of Conversations on Truth Social
by: Ameen, Fathima, et al.
Published: (2026)

Evolution of AI in Education: Agentic Workflows
by: Kamalov, Firuz, et al.
Published: (2025)

The ASIR Courage Model: A Phase-Dynamic Framework for Truth Transitions in Human and AI Systems
by: Kim, Hyo Jin
Published: (2026)

Automatic Large Language Models Creation of Interactive Learning Lessons
by: Lin, Jionghao, et al.
Published: (2025)

Predicting Learning Performance with Large Language Models: A Study in Adult Literacy
by: Zhang, Liang, et al.
Published: (2024)

Beyond Tool Adoption: A Practical Five-Stage Developmental Continuum for AI Literacy in Higher Education
by: Liu, J. Paul, et al.
Published: (2026)

Rethinking AI Evaluation in Education: The TEACH-AI Framework and Benchmark for Generative AI Assistants
by: Ding, Shi, et al.
Published: (2025)

Enhancing the De-identification of Personally Identifiable Information in Educational Data
by: Ji, Zilyu, et al.
Published: (2025)

Beyond Accuracy: Rethinking Hallucination and Regulatory Response in Generative AI
by: Li, Zihao, et al.
Published: (2025)

Annotating the Chain-of-Thought: A Behavior-Labeled Dataset for AI Safety
by: Menke, Antonio-Gabriel Chacón, et al.
Published: (2025)

Comparing Few-Shot Prompting of GPT-4 LLMs with BERT Classifiers for Open-Response Assessment in Tutor Equity Training
by: Kakarla, Sanjit, et al.
Published: (2025)