Cover Image

ConSiDERS-The-Human Evaluation Framework: Rethinking Human Evaluation for Generative Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Elangovan, Aparna, Liu, Ling, Xu, Lei, Bodapati, Sravan, Roth, Dan
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Beyond correlation: The Impact of Human Uncertainty in Measuring the Effectiveness of Automatic Evaluation and LLM-as-a-Judge
by: Elangovan, Aparna, et al.
Published: (2024)

SocREval: Large Language Models with the Socratic Method for Reference-Free Reasoning Evaluation
by: He, Hangfeng, et al.
Published: (2023)

Evaluating Creative Short Story Generation in Humans and Large Language Models
by: Ismayilzada, Mete, et al.
Published: (2024)

Rethinking Human Preference Evaluation of LLM Rationales
by: Li, Ziang, et al.
Published: (2025)

A Framework for Human Evaluation of Large Language Models in Healthcare Derived from Literature Review
by: Tam, Thomas Yu Chow, et al.
Published: (2024)

Salient Information Prompting to Steer Content in Prompt-based Abstractive Summarization
by: Xu, Lei, et al.
Published: (2024)

Rethinking Toxicity Evaluation in Large Language Models: A Multi-Label Perspective
by: Kou, Zhiqiang, et al.
Published: (2025)

ConSCompF: Consistency-focused Similarity Comparison Framework for Generative Large Language Models
by: Karev, Alexey, et al.
Published: (2025)

GenRES: Rethinking Evaluation for Generative Relation Extraction in the Era of Large Language Models
by: Jiang, Pengcheng, et al.
Published: (2024)

The Generative Energy Arena (GEA): Incorporating Energy Awareness in Large Language Model (LLM) Human Evaluations
by: Arriaga, Carlos, et al.
Published: (2025)

PROXYQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models
by: Tan, Haochen, et al.
Published: (2024)

Evaluating Large Language Models with Human Feedback: Establishing a Swedish Benchmark
by: Moell, Birger
Published: (2024)

SLMEval: Entropy-Based Calibration for Human-Aligned Evaluation of Large Language Models
by: Daynauth, Roland, et al.
Published: (2025)

ReEval: Automatic Hallucination Evaluation for Retrieval-Augmented Large Language Models via Transferable Adversarial Attacks
by: Yu, Xiaodong, et al.
Published: (2023)

PokeLLMon: A Human-Parity Agent for Pokemon Battles with Large Language Models
by: Hu, Sihao, et al.
Published: (2024)

An In-depth Evaluation of Large Language Models in Sentence Simplification with Error-based Human Assessment
by: Wu, Xuanxin, et al.
Published: (2024)

Elsevier Arena: Human Evaluation of Chemistry/Biology/Health Foundational Large Language Models
by: Thorne, Camilo, et al.
Published: (2024)

Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators
by: Liu, Yinhong, et al.
Published: (2024)

ConDABench: Interactive Evaluation of Language Models for Data Analysis
by: Dutta, Avik, et al.
Published: (2025)

LAWCAT: Efficient Distillation from Quadratic to Linear Attention with Convolution across Tokens for Long Context Modeling
by: Liu, Zeyu, et al.
Published: (2025)

LalaEval: A Holistic Human Evaluation Framework for Domain-Specific Large Language Models
by: Sun, Chongyan, et al.
Published: (2024)

Towards Unifying Evaluation of Counterfactual Explanations: Leveraging Large Language Models for Human-Centric Assessments
by: Domnich, Marharyta, et al.
Published: (2024)

A Proposed S.C.O.R.E. Evaluation Framework for Large Language Models : Safety, Consensus, Objectivity, Reproducibility and Explainability
by: Tan, Ting Fang, et al.
Published: (2024)

ConCodeEval: Evaluating Large Language Models for Code Constraints in Domain-Specific Languages
by: Kammakomati, Mehant, et al.
Published: (2024)

CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation
by: Ke, Pei, et al.
Published: (2023)

AutoMetrics: Approximate Human Judgements with Automatically Generated Evaluators
by: Ryan, Michael J., et al.
Published: (2025)

Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Sets
by: Gupta, Vatsal, et al.
Published: (2023)

HREF: Human Response-Guided Evaluation of Instruction Following in Language Models
by: Lyu, Xinxi, et al.
Published: (2024)

Adaptive Multi-Agent Response Refinement in Conversational Systems
by: Jeong, Soyeong, et al.
Published: (2025)

LLMs Judge Themselves: A Game-Theoretic Framework for Human-Aligned Evaluation
by: Yang, Gao, et al.
Published: (2025)

Can LLMs Narrate Tabular Data? An Evaluation Framework for Natural Language Representations of Text-to-SQL System Outputs
by: Singh, Jyotika, et al.
Published: (2025)

Developing A Framework to Support Human Evaluation of Bias in Generated Free Response Text
by: Healey, Jennifer, et al.
Published: (2025)

HumBEL: A Human-in-the-Loop Approach for Evaluating Demographic Factors of Language Models in Human-Machine Conversations
by: Sicilia, Anthony, et al.
Published: (2023)

The Effectiveness of Style Vectors for Steering Large Language Models: A Human Evaluation
by: Diallo, Diaoulé, et al.
Published: (2026)

Hierarchical Prompting Taxonomy: A Universal Evaluation Framework for Large Language Models Aligned with Human Cognitive Principles
by: Budagam, Devichand, et al.
Published: (2024)

Evaluating Morphological Compositional Generalization in Large Language Models
by: Ismayilzada, Mete, et al.
Published: (2024)

What is the best model? Application-driven Evaluation for Large Language Models
by: Lian, Shiguo, et al.
Published: (2024)

Evaluating Large Language Models for Abstract Evaluation Tasks: An Empirical Study
by: Liu, Yinuo, et al.
Published: (2026)

FineMath: A Fine-Grained Mathematical Evaluation Benchmark for Chinese Large Language Models
by: Liu, Yan, et al.
Published: (2024)

Evaluating Quantized Large Language Models
by: Li, Shiyao, et al.
Published: (2024)