Cover Image

Eval Factsheets: A Structured Framework for Documenting AI Evaluations

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bordes, Florian, Ross, Candace, Kao, Justine T, Spiliopoulou, Evangelia, Williams, Adina
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

What's in Common? Multimodal Models Hallucinate When Reasoning Across Scenes
by: Ross, Candace, et al.
Published: (2025)

Improving Model Evaluation using SMART Filtering of Benchmark Datasets
by: Gupta, Vipul, et al.
Published: (2024)

VoxelCodeBench: Benchmarking 3D World Modeling Through Code Generation
by: Zheng, Yan, et al.
Published: (2026)

IntPhys 2: Benchmarking Intuitive Physics Understanding In Complex Synthetic Environments
by: Bordes, Florian, et al.
Published: (2025)

Balancing Classification and Calibration Performance in Decision-Making LLMs via Calibration Aware Reinforcement Learning
by: Yaldiz, Duygu Nur, et al.
Published: (2026)

Detecting Training Data of Large Language Models via Expectation Maximization
by: Kim, Gyuwan, et al.
Published: (2024)

GraphEval: A Lightweight Graph-Based LLM Framework for Idea Evaluation
by: Feng, Tao, et al.
Published: (2025)

EvalQReason: A Framework for Step-Level Reasoning Evaluation in Large Language Models
by: Freja, Shaima Ahmad, et al.
Published: (2026)

Brittlebench: Quantifying LLM robustness via prompt sensitivity
by: Romanou, Angelika, et al.
Published: (2026)

SynthEval: A Framework for Detailed Utility and Privacy Evaluation of Tabular Synthetic Data
by: Lautrup, Anton Danholt, et al.
Published: (2024)

What makes a good metric? Evaluating automatic metrics for text-to-image consistency
by: Ross, Candace, et al.
Published: (2024)

Cer-Eval: Certifiable and Cost-Efficient Evaluation Framework for LLMs
by: Wang, Ganghua, et al.
Published: (2025)

Measuring Déjà vu Memorization Efficiently
by: Kokhlikyan, Narine, et al.
Published: (2025)

AlphaEval: A Comprehensive and Efficient Evaluation Framework for Formula Alpha Mining
by: Ding, Hongjun, et al.
Published: (2025)

GraphEval: A Knowledge-Graph Based LLM Hallucination Evaluation Framework
by: Sansford, Hannah, et al.
Published: (2024)

ProEval: Proactive Failure Discovery and Efficient Performance Estimation for Generative AI Evaluation
by: Huang, Yizheng, et al.
Published: (2026)

OpenCarbonEval: A Unified Carbon Emission Estimation Framework in Large-Scale AI Models
by: Yu, Zhaojian, et al.
Published: (2024)

Transformers Can Navigate Mazes With Multi-Step Prediction
by: Nolte, Niklas, et al.
Published: (2024)

GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation
by: Aksu, Taha, et al.
Published: (2024)

Compositional learning of functions in humans and machines
by: Zhou, Yanli, et al.
Published: (2024)

DivLogicEval: A Framework for Benchmarking Logical Reasoning Evaluation in Large Language Models
by: Chung, Tsz Ting, et al.
Published: (2025)

InfiCoEvalChain: A Blockchain-Based Decentralized Framework for Collaborative LLM Evaluation
by: Yang, Yifan, et al.
Published: (2026)

Revisiting Gradient Staleness: Evaluating Distance Metrics for Asynchronous Federated Learning Aggregation
by: Wilhelm, Patrick, et al.
Published: (2026)

StructEval: Deepen and Broaden Large Language Model Assessment via Structured Evaluation
by: Cao, Boxi, et al.
Published: (2024)

ThermEval: A Structured Benchmark for Evaluation of Vision-Language Models on Thermal Imagery
by: Shrivastava, Ayush, et al.
Published: (2026)

Eval-PPO: Building an Efficient Threat Evaluator Using Proximal Policy Optimization
by: Sun, Wuzhou, et al.
Published: (2025)

Do different prompting methods yield a common task representation in language models?
by: Davidson, Guy, et al.
Published: (2025)

A Context-Aware Dual-Metric Framework for Confidence Estimation in Large Language Models
by: Yuan, Mingruo, et al.
Published: (2025)

FROAV: A Framework for RAG Observation and Agent Verification -- Lowering the Barrier to LLM Agent Research
by: Lin, Tzu-Hsuan, et al.
Published: (2026)

DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA
by: Patel, Aman, et al.
Published: (2024)

QualEval: Qualitative Evaluation for Model Improvement
by: Murahari, Vishvak, et al.
Published: (2023)

AsymmetryZero: A Framework for Operationalizing Human Expert Preferences as Semantic Evals
by: Looram, Tadhg, et al.
Published: (2026)

MMCircuitEval: A Comprehensive Multimodal Circuit-Focused Benchmark for Evaluating LLMs
by: Zhao, Chenchen, et al.
Published: (2025)

MicroEvoEval: A Systematic Evaluation Framework for Image-Based Microstructure Evolution Prediction
by: Zhang, Qinyi, et al.
Published: (2025)

RxEval: A Prescription-Level Benchmark for Evaluating LLM Medication Recommendation
by: Chen, Shuhao, et al.
Published: (2026)

Privacy-Preserving Distributed Learning in IoT Systems: A Unified Threat Model and Evaluation Framework
by: Cartmell, John, et al.
Published: (2026)

Weisfeiler and Leman Go Measurement Modeling: Probing the Validity of the WL Test
by: Subramonian, Arjun, et al.
Published: (2023)

Feedback-guided Data Synthesis for Imbalanced Classification
by: Hemmat, Reyhane Askari, et al.
Published: (2023)

SparseEval: Efficient Evaluation of Large Language Models by Sparse Optimization
by: Zhang, Taolin, et al.
Published: (2026)

A Shortcut-aware Video-QA Benchmark for Physical Understanding via Minimal Video Pairs
by: Krojer, Benno, et al.
Published: (2025)