Cover Image

PredictaBoard: Benchmarking LLM Score Predictability

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Pacchiardi, Lorenzo, Voudouris, Konstantinos, Slater, Ben, Martínez-Plumed, Fernando, Hernández-Orallo, José, Zhou, Lexin, Schellaert, Wout
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Measuring Data Science Automation: A Survey of Evaluation Tools for AI Assistants and Agents
by: Testini, Irene, et al.
Published: (2025)

100 instances is all you need: predicting the success of a new LLM on unseen data by testing on a few instances
by: Pacchiardi, Lorenzo, et al.
Published: (2024)

Leaving the barn door open for Clever Hans: Simple features predict LLM benchmark answers
by: Pacchiardi, Lorenzo, et al.
Published: (2024)

Predictable Artificial Intelligence
by: Zhou, Lexin, et al.
Published: (2023)

What should an AI assessor optimise for?
by: Romero-Alvarado, Daniel, et al.
Published: (2025)

Confident Rankings with Fewer Items: Adaptive LLM Evaluation with Continuous Scores
by: Balkır, Esma, et al.
Published: (2026)

The Animal-AI Environment: A Virtual Laboratory For Comparative Cognition and Artificial Intelligence Research
by: Voudouris, Konstantinos, et al.
Published: (2023)

Paradigms of AI Evaluation: Mapping Goals, Methodologies and Culture
by: Burden, John, et al.
Published: (2025)

General Scales Unlock AI Evaluation with Explanatory and Predictive Power
by: Zhou, Lexin, et al.
Published: (2025)

No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probes
by: Cencerrado, Iván Vicente Moreno, et al.
Published: (2025)

Measuring What AI Systems Might Do: Towards A Measurement Science in AI
by: Voudouris, Konstantinos, et al.
Published: (2026)

Cognitive Science-Inspired Evaluation of Core Capabilities for Object Understanding in AI
by: Rutar, Danaja, et al.
Published: (2025)

An LLM Feature-based Framework for Dialogue Constructiveness Assessment
by: Zhou, Lexin, et al.
Published: (2024)

Inferring Capabilities from Task Performance with Bayesian Triangulation
by: Burden, John, et al.
Published: (2023)

Capabilities Ain't All You Need: Measuring Propensities in AI
by: Romero-Alvarado, Daniel, et al.
Published: (2026)

Conversational Complexity for Assessing Risk in Large Language Models
by: Burden, John, et al.
Published: (2024)

Exploring Major Transitions in the Evolution of Biological Cognition With Artificial Neural Networks
by: Voudouris, Konstantinos, et al.
Published: (2025)

Psychometric Personality Shaping Modulates Capabilities and Safety in Language Models
by: Fitz, Stephen, et al.
Published: (2025)

A little less conversation, a little more action, please: Investigating the physical common-sense of LLMs in a 3D embodied environment
by: Mecattaf, Matteo G., et al.
Published: (2024)

metabench -- A Sparse Benchmark of Reasoning and Knowledge in Large Language Models
by: Kipnis, Alex, et al.
Published: (2024)

Silicon Bureaucracy and AI Test-Oriented Education: Contamination Sensitivity and Score Confidence in LLM Benchmarks
by: Song, Yiliang, et al.
Published: (2026)

AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents
by: Ma, Chang, et al.
Published: (2024)

Seven simple steps for log analysis in AI systems
by: Dubois, Magda, et al.
Published: (2026)

JudgeBoard: Benchmarking and Enhancing Small Language Models for Reasoning Evaluation
by: Bi, Zhenyu, et al.
Published: (2025)

Multi-agent AI systems outperform human teams in creativity
by: Hu, Tiancheng, et al.
Published: (2026)

TripScore: Benchmarking and rewarding real-world travel planning with fine-grained evaluation
by: Qu, Yincen, et al.
Published: (2025)

DiffLM: Controllable Synthetic Data Generation via Diffusion Language Models
by: Zhou, Ying, et al.
Published: (2024)

Benchmark Profiling: Mechanistic Diagnosis of LLM Benchmarks
by: Kim, Dongjun, et al.
Published: (2025)

Rationale Behind Essay Scores: Enhancing S-LLM's Multi-Trait Essay Scoring with Rationale Generated by LLMs
by: Chu, SeongYeub, et al.
Published: (2024)

Vocabulary Dropout for Curriculum Diversity in LLM Co-Evolution
by: Dineen, Jacob, et al.
Published: (2026)

Is Your LLM Really Mastering the Concept? A Multi-Agent Benchmark
by: Xu, Shuhang, et al.
Published: (2025)

A Statistical Hypothesis Testing Framework for Data Misappropriation Detection in Large Language Models
by: Cai, Yinpeng, et al.
Published: (2025)

Knowledge Distillation of LLM for Automatic Scoring of Science Education Assessments
by: Latif, Ehsan, et al.
Published: (2023)

Improve LLM-based Automatic Essay Scoring with Linguistic Features
by: Hou, Zhaoyi Joey, et al.
Published: (2025)

Pair2Score: Pairwise-to-Absolute Transfer for LLM-Based Essay Scoring
by: Hallaç, İbrahim Rıza, et al.
Published: (2026)

Bringing Comparative Cognition To Computers
by: Voudouris, Konstantinos, et al.
Published: (2025)

Beyond Pointwise Scores: Decomposed Criteria-Based Evaluation of LLM Responses
by: Yu, Fangyi, et al.
Published: (2025)

PatentScore: Multi-dimensional Evaluation of LLM-Generated Patent Claims
by: Yoo, Yongmin, et al.
Published: (2025)

Same Meaning, Different Scores: Lexical and Syntactic Sensitivity in LLM Evaluation
by: Kostić, Bogdan, et al.
Published: (2026)

Contrastive Decoding Mitigates Score Range Bias in LLM-as-a-Judge
by: Fujinuma, Yoshinari
Published: (2025)