Image de couverture de livre

Criterion-referenceability determines LLM-as-a-judge validity across physics assessment formats

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yeadon, Will, Hardy, Tom, Mackay, Paul, Agra, Elise
Format:	Preprint
Publié:	2026
Sujets:	Physics Education Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Evaluating AI and Human Authorship Quality in Academic Writing through Physics Essays
par: Yeadon, Will, et autres
Publié: (2024)

The Impact of AI in Physics Education: A Comprehensive Review from GCSE to University Levels
par: Yeadon, Will, et autres
Publié: (2023)

Evaluating NLP Embedding Models for Handling Science-Specific Symbolic Expressions in Student Texts
par: Bleckmann, Tom, et autres
Publié: (2025)

Stan: An LLM-based thermodynamics course assistant
par: Furst, Eric M., et autres
Publié: (2026)

From Canonical to Complex: Benchmarking LLM Capabilities in Undergraduate Thermodynamics
par: Geißler, Anna, et autres
Publié: (2025)

A comparison of Human, GPT-3.5, and GPT-4 Performance in a University-Level Coding Course
par: Yeadon, Will, et autres
Publié: (2024)

The Honorific Effect: Exploring the Impact of Japanese Linguistic Formalities on AI-Generated Physics Explanations
par: Sato, Keisuke
Publié: (2024)

Scalable and consistent few-shot classification of survey responses using text embeddings
par: Mjaaland, Jonas Timmann, et autres
Publié: (2025)

Small Language Models Reshape Higher Education: Courses, Textbooks, and Teaching
par: Zhang, Jian, et autres
Publié: (2025)

Daily and Weekly Periodicity in Large Language Model Performance and Its Implications for Research
par: Tschisgale, Paul, et autres
Publié: (2026)

Self-rationalization improves LLM as a fine-grained judge
par: Trivedi, Prapti, et autres
Publié: (2024)

An LLM-as-a-judge Approach for Scalable Gender-Neutral Translation Evaluation
par: Piergentili, Andrea, et autres
Publié: (2025)

Understanding interaction network formation across instructional contexts in remote physics courses
par: Sundstrom, Meagan, et autres
Publié: (2022)

Curiosity-Driven LLM-as-a-judge for Personalized Creative Judgment
par: Kumar, Vanya Bannihatti, et autres
Publié: (2025)

From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge
par: Li, Dawei, et autres
Publié: (2024)

Who judges the judges? Governance from metrics: a runtime framework for continuous LLM compliance monitoring
par: Dussert, Jehanne
Publié: (2026)

When Wording Steers the Evaluation: Framing Bias in LLM judges
par: Hwang, Yerin, et autres
Publié: (2026)

Scaling Physical Reasoning with the PHYSICS Dataset
par: Zheng, Shenghe, et autres
Publié: (2025)

Assessing Large Language Models in Mechanical Engineering Education: A Study on Mechanics-Focused Conceptual Understanding
par: Tian, Jie, et autres
Publié: (2024)

Translating scientific Latin texts with artificial intelligence: the works of Euler and contemporaries
par: Bistafa, Sylvio R.
Publié: (2023)

Mastering Olympiad-Level Physics with Artificial Intelligence
par: Jian, Dong-Shan, et autres
Publié: (2025)

Dissecting Physics Reasoning in Small Language Models: A Multi-Dimensional Analysis from an Educational Perspective
par: Scaria, Nicy, et autres
Publié: (2025)

Preference Leakage: A Contamination Problem in LLM-as-a-judge
par: Li, Dawei, et autres
Publié: (2025)

Efficacy of a hybrid take home and in class summative assessment for the postsecondary physics classroom
par: Stonaha, Paul, et autres
Publié: (2024)

LLM-as-a-qualitative-judge: automating error analysis in natural language generation
par: Chirkova, Nadezhda, et autres
Publié: (2025)

From Calculation to Adjudication: Examining LLM judges on Mathematical Reasoning Tasks
par: Stephan, Andreas, et autres
Publié: (2024)

QISCIT: A validated concept inventory assessment for quantum information science
par: Durkin, Kelley, et autres
Publié: (2025)

How Hypocritical Is Your LLM judge? Listener-Speaker Asymmetries in the Pragmatic Competence of Large Language Models
par: Sieker, Judith, et autres
Publié: (2026)

Concurrent Criterion Validation of a Validity Screen for LLM Confidence Signals via Selective Prediction
par: Cacioli, Jon-Paul
Publié: (2026)

Development and validation of a conceptual multiple-choice survey instrument to assess student understanding of introductory thermodynamics
par: Brundage, Mary Jane, et autres
Publié: (2024)

Multi-Agent LLM Judge: automatic personalized LLM judge design for evaluating natural language generation applications
par: Cao, Hongliu, et autres
Publié: (2025)

Online antisemitism across platforms
par: De Smedt, Tom
Publié: (2021)

Criterion Validity of LLM-as-Judge for Business Outcomes in Conversational Commerce
par: Chen, Liang, et autres
Publié: (2026)

Comparing student performance on a multi-attempt asynchronous assessment to a single-attempt synchronous assessment in introductory level physics
par: Frederick, Emily, et autres
Publié: (2024)

Overview of couplet scoring in content-focused physics assessments
par: Vignal, Michael, et autres
Publié: (2023)

Calibrating MLLM-as-a-judge via Multimodal Bayesian Prompt Ensembles
par: Slyman, Eric, et autres
Publié: (2025)

Impact of Criterion-Based Reflection on Prospective Physics Teachers' Perceptions of ChatGPT-Generated Content
par: Sadidi, Farahnaz, et autres
Publié: (2024)

Evaluating recognition and recall formats of social network surveys in physics education research
par: Sundstrom, Meagan, et autres
Publié: (2025)

Society's educational debts in biology, chemistry, and physics across race, gender, and class
par: Van Dusen, Ben, et autres
Publié: (2024)

Investigating peer recognition across an introductory physics sequence: Do first impressions last?
par: Sundstrom, Meagan, et autres
Publié: (2023)