Imagen de Portada

The Scaling Law of Evaluation Failure: Why Simple Averaging Collapses Under Data Sparsity and Item Difficulty Gaps, and How Item Response Theory Recovers Ground Truth Across Domains

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Kang, Jung Min
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

The Impact of Item-Writing Flaws on Difficulty and Discrimination in Item Response Theory
por: Schmucker, Robin, et al.
Publicado: (2025)

RIDE: Difficulty Evolving Perturbation with Item Response Theory for Mathematical Reasoning
por: Li, Xinyuan, et al.
Publicado: (2025)

Prediction of Item Difficulty for Reading Comprehension Items by Creation of Annotated Item Repository
por: Kapoor, Radhika, et al.
Publicado: (2025)

Fairness Evaluation with Item Response Theory
por: Xu, Ziqi, et al.
Publicado: (2024)

SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction
por: Scarlatos, Alexander, et al.
Publicado: (2025)

Item Response Theory Analysis of the Classroom Level Adversity Scale
por: Allyson L. Hayward, et al.
Publicado: (2026)

Estimating LLM Grading Ability and Response Difficulty in Automatic Short Answer Grading via Item Response Theory
por: Cong, Longwei, et al.
Publicado: (2026)

Estimating Heterogeneous Treatment Effects with Item-Level Outcome Data: Insights from Item Response Theory
por: Gilbert, Joshua B., et al.
Publicado: (2024)

Robust Estimation of Item Parameters via Divergence Measures in Item Response Theory
por: Itaya, Yuki, et al.
Publicado: (2025)

On Truthful Item-Acquiring Mechanisms for Reward Maximization
por: Shan, Liang, et al.
Publicado: (2024)

Truthful Matching with Online Items and Offline Agents
por: Feldman, Michal, et al.
Publicado: (2022)

Auditing LLM Benchmarks with Item Response Theory
por: Land, Sander, et al.
Publicado: (2026)

Sparse Bayesian Multidimensional Item Response Theory
por: Li, Jiguang, et al.
Publicado: (2023)

Scalable Learning of Item Response Theory Models
por: Frick, Susanne, et al.
Publicado: (2024)

Estimating Item Difficulty with Large Language Models as Experts
por: Kolesnikova, Diana, et al.
Publicado: (2026)

Using Vision + Language Models to Predict Item Difficulty
por: Khan, Samin
Publicado: (2026)

Identifying Word‐Problem Difficulty: An Item Response Analysis of an Additive Word‐Problem Screener
por: Alison M. Hardy, et al.
Publicado: (2025)

Estimating Racial and Ethnic Healthcare Quality Disparities Using Exploratory Item Response Theory and Latent Class Item Response Theory Models
por: Normand, Sharon-Lise, et al.
Publicado: (2023)

Known-Item Search Failure in an OPAC.
por: Dwyer, Catherine M., et al.
Publicado: (1991)

Modeling Local Item Dependence in Cloze and Reading Comprehension Test Items Using Testlet Response Theory
por: Purya Baghaei
Publicado: (2016)

An Item Response Tree Model for Items with Multiple‐Choice and Constructed‐Response Parts
por: Junhuan Wei, et al.
Publicado: (2024)

Multidimensional Item Response Theory in the Style of Collaborative Filtering
por: Bergner, Yoav, et al.
Publicado: (2023)

Synthesizing the Ability in Multidimensional Item Response Theory Models
por: ÁLVARO MAURICIO MONTENEGRO DÍAZ
Publicado: (2010)

Difficulty as a Proxy for Measuring Intrinsic Cognitive Load Item
por: Cai, Minghao, et al.
Publicado: (2025)

Crowdsourcing with Difficulty: A Bayesian Rating Model for Heterogeneous Items
por: Han, Seong Woo, et al.
Publicado: (2024)

Introducing Flexible Monotone Multiple Choice Item Response Theory Models and Bit Scales
por: Wallmark, Joakim, et al.
Publicado: (2024)

Text-Based Approaches to Item Difficulty Modeling in Large-Scale Assessments: A Systematic Review
por: Peters, Sydney, et al.
Publicado: (2025)

Simultaneous Detection of Compromised Items and Examinees with Item Preknowledge in Online Assessments Using Response Time Data
por: Cengiz Zopluoglu
Publicado: (2026)

Correcting Human Labels for Rater Effects in AI Evaluation: An Item Response Theory Approach
por: Casabianca, Jodi M., et al.
Publicado: (2026)

Evaluating Cross-Modal Reasoning Ability and Problem Characteristics with Multimodal Item Response Theory
por: Uebayashi, Shunki, et al.
Publicado: (2026)

Asymptotic Standard Errors for Reliability Coefficients in Item Response Theory
por: Sung, Youjin, et al.
Publicado: (2025)

UnibucLLM: Harnessing LLMs for Automated Prediction of Item Difficulty and Response Time for Multiple-Choice Questions
por: Rogoz, Ana-Cristina, et al.
Publicado: (2024)

Item Selection Strategies Combining Response Time with Item Exposure Control in MCAT
por: Yuwei Luo, et al.
Publicado: (2026)

Development of a Japanese Version of the Creative Mindset Scale (CMS) Using Item Response Theory
por: Chiaki Ishiguro, et al.
Publicado: (2024)

Using Item Response Theory Models in Scaling Severity Scores in Alcohol Research—A Tutorial
por: Daniel Schulze, et al.
Publicado: (2025)

Refining Scale Measurement: Reassessing Oral Impacts on Daily Performances Properties With Item Response Theory
por: Roger Keller Celeste, et al.
Publicado: (2025)

A Doubly Latent Space Joint Model for Local Item and Person Dependence in the Analysis of Item Response Data
por: Jin, Ick Hoon, et al.
Publicado: (2016)

Measuring Competency, Not Performance: Item-Aware Evaluation Across Medical Benchmarks
por: Luo, Zhimeng, et al.
Publicado: (2025)

Differences in Item Discrimination of the 25‐Question Geriatric Locomotive Function Scale Between Younger and Middle‐Aged Adults and Older Adults: An Analysis Using the Item Response Theory
por: Shunsuke Yamashina, et al.
Publicado: (2026)

The Development and Evaluation of an Alternative Abbreviated Nutrition Risk Screening Tool Using Item Response Theory
por: Meagan S. Jackson, et al.
Publicado: (2026)