Copertina

Investigation of the Inter-Rater Reliability between Large Language Models and Human Raters in Qualitative Analysis

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Borse, Nikhil Sanjay, Subramaniam, Ravishankar Chatta, Rebello, N. Sanjay
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Physics Education
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Investigating the Design-Science Connection in a multi-week Engineering Design (ED)-based introductory physics laboratory task
di: Subramaniam, Ravishankar Chatta, et al.
Pubblicazione: (2024)

Applying a STEM Ways of Thinking Framework for Student-generated Engineering Design-based Physics Problems
di: Subramaniam, Ravishankar Chatta, et al.
Pubblicazione: (2025)

Presenting a STEM Ways of Thinking Framework for Engineering Design-based Physics Problems
di: Subramaniam, Ravishankar Chatta, et al.
Pubblicazione: (2024)

Using an LLM to Investigate Students' Explanations on Conceptual Physics Questions
di: Savage, Sean, et al.
Pubblicazione: (2025)

Students' Perceptions to a Large Language Model's Generated Feedback and Scores of Argumentation Essays
di: Allen, Winter, et al.
Pubblicazione: (2025)

AI Reasoning Models for Problem Solving in Physics
di: Bralin, Amir, et al.
Pubblicazione: (2025)

Assessing Physics Students' Scientific Argumentation using Natural Language Processing
di: Allen, Winter, et al.
Pubblicazione: (2025)

Sense of Belonging and Intent to Persist: Mediating Role of Motivation and Moderating Role of Gender in Physics and Astronomy Graduate Students
di: Sarkar, Swagata, et al.
Pubblicazione: (2025)

Feedback That Clicks: Introductory Physics Students' Valued Features in AI Feedback Generated From Self-Crafted and Engineered Prompts
di: Sirnoorkar, Amogh, et al.
Pubblicazione: (2025)

From Self-Crafted to Engineered Prompts: Student Evaluations of AI-Generated Feedback in Introductory Physics
di: Sirnoorkar, Amogh, et al.
Pubblicazione: (2025)

Exploring the Feasibility of Employing a Hybrid Machine Learning Method to Unpack Student Reasoning Patterns in Physics Essays
di: Allen, Winter, et al.
Pubblicazione: (2025)

Cognitive Load and Situational Interest in Physics Laboratories: A Comparative Study Across Three Instructional Modalities
di: Hamed, Razan, et al.
Pubblicazione: (2026)

IRT Observed‐Score Equating for Rater‐Mediated Assessments Using a Hierarchical Rater Model
di: Tong Wu, et al.
Pubblicazione: (2025)

Dual-Role Dynamics in Prompting: Elementary Pre-service Teachers' AI Prompting Strategies for Representational Choices
di: Hamed, Razan, et al.
Pubblicazione: (2025)

Help or Hype? Students' Engagement and Perception of Using AI to Solve Physics Problems
di: Mirza, Qurat-ul-Ann, et al.
Pubblicazione: (2025)

Analyzing Undergraduate Problem-Solving in Physics Through Interaction With an AI Chatbot
di: Hashmi, Syed Furqan Abbas, et al.
Pubblicazione: (2025)

Relating visual attention and learning in an online instructional physics module
di: Hamed, Razan, et al.
Pubblicazione: (2026)

Using Computational Physics Essays to Facilitate Engineering Students' Computational Thinking
di: Savage, Sean, et al.
Pubblicazione: (2026)

Examining Student and AI Generated Personalized Analogies in Introductory Physics
di: Sirnoorkar, Amogh, et al.
Pubblicazione: (2025)

Evaluating Rater Effects of Large Language Models in Automated Essay Scoring: GPT, Claude, Gemini, and DeepSeek
di: Hong Jiao, et al.
Pubblicazione: (2026)

Intra‐ and Inter‐Rater Reliability of Tongue Physical Function Assessments in Children
di: Pierre Cnockaert, et al.
Pubblicazione: (2026)

Longitudinal Cross‐Classified Item Response Theory Model: Application to Longitudinal Rater‐Mediated Assessment
di: Yun Kyung Kim, et al.
Pubblicazione: (2026)

Principled Evaluation with Human Labels: One Rater at a Time and Rater Equivalence
di: Resnick, Paul, et al.
Pubblicazione: (2021)

Student and AI responses to physics problems examined through the lenses of sensemaking and mechanistic reasoning
di: Sirnoorkar, Amogh, et al.
Pubblicazione: (2024)

Comparison of Scoring Rationales Between Large Language Models and Human Raters
di: Hua, Haowei, et al.
Pubblicazione: (2025)

Supporting Human Raters with the Detection of Harmful Content using Large Language Models
di: Thomas, Kurt, et al.
Pubblicazione: (2024)

Inter‐Rater Reliability of Video Based Sonographic Assessment of Lower Extremity Tendon and Fascia Abnormalities
di: Derek C. Stokes, et al.
Pubblicazione: (2025)

Multi-Rater Calibrated Segmentation Models
di: Riera-Marín, Meritxell, et al.
Pubblicazione: (2026)

ALICE: Combining Feature Selection and Inter-Rater Agreeability for Machine Learning Insights
di: Anasashvili, Bachana, et al.
Pubblicazione: (2024)

Picky Eaters Make For Better Raters
di: Stoikov, Sasha, et al.
Pubblicazione: (2024)

Intra‐ and Inter‐Rater Reliability of Linear and Nonlinear Measures of Short‐Term Heart Rate Variability Following Combat‐Related Traumatic Injury
di: Rabeea Maqsood, et al.
Pubblicazione: (2024)

Through the Judge's Eyes: Inferred Thinking Traces Improve Reliability of LLM Raters
di: Zhang, Xingjian, et al.
Pubblicazione: (2025)

Inter‐Rater Reliability of Active Cervical Range of Motion Measured With Digital Goniometer in Asymptomatic University‐Level American Football Athletes
di: Mathieu Lanoue, et al.
Pubblicazione: (2026)

Comparing Human and AI Rater Effects Using the Many-Facet Rasch Model
di: Jiao, Hong, et al.
Pubblicazione: (2025)

Assessing Student Errors in Experimentation Using Artificial Intelligence and Large Language Models: A Comparative Study with Human Raters
di: Bewersdorff, Arne, et al.
Pubblicazione: (2023)

DataRater: Meta-Learned Dataset Curation
di: Calian, Dan A., et al.
Pubblicazione: (2025)

Community Notes are Vulnerable to Rater Bias and Manipulation
di: Truong, Bao Tran, et al.
Pubblicazione: (2025)

The Determinants of ESG Ratings: Rater Ownership Matters
di: Dragon Yongjun Tang, et al.
Pubblicazione: (2025)

SkillRater: Untangling Capabilities in Multimodal Data
di: Sahi, Naveen, et al.
Pubblicazione: (2026)

Asymptotic Confidence Interval, Sample Size Formulas and Comparison Test for the Agreement Intra‐Class Correlation Coefficient in Inter‐Rater Reliability Studies
di: Abderrahmane Bourredjem, et al.
Pubblicazione: (2024)