Cover Image

Towards Acyclic Preference Evaluation of Language Models via Multiple Evaluators

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hu, Zhengyu, Zhang, Jieyu, Xiong, Zhihan, Ratner, Alexander, Ding, Kaize, Krishna, Ranjay
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Explaining Length Bias in LLM-Based Preference Evaluations
by: Hu, Zhengyu, et al.
Published: (2024)

The Strongest Teacher Is Not Always the Best Teacher: Student-Centric Answer Selection
by: Hu, Zhengyu, et al.
Published: (2026)

GNN-as-Judge: Unleashing the Power of LLMs for Graph Learning with GNN Feedback
by: Xu, Ruiyao, et al.
Published: (2026)

SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models
by: Wang, Xiaoxuan, et al.
Published: (2023)

Population-Aligned Persona Generation for LLM-based Social Simulation
by: Hu, Zhengyu, et al.
Published: (2025)

RealCritic: Towards Effectiveness-Driven Evaluation of Language Model Critiques
by: Tang, Zhengyang, et al.
Published: (2025)

Answer Matching Outperforms Multiple Choice for Language Model Evaluation
by: Chandak, Nikhil, et al.
Published: (2025)

Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators
by: Liu, Yinhong, et al.
Published: (2024)

Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps
by: Chuang, Yung-Sung, et al.
Published: (2024)

Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
by: Hsieh, Cheng-Yu, et al.
Published: (2024)

Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
by: Wu, Junkang, et al.
Published: (2024)

Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration?
by: Zhang, Pingyue, et al.
Published: (2026)

Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluation
by: Vu, Tu, et al.
Published: (2024)

MENLO: From Preferences to Proficiency -- Evaluating and Modeling Native-like Quality Across 47 Languages
by: Whitehouse, Chenxi, et al.
Published: (2025)

Are LLM Evaluators Really Narcissists? Sanity Checking Self-Preference Evaluations
by: Roytburg, Dani, et al.
Published: (2026)

Towards Multilingual LLM Evaluation for European Languages
by: Thellmann, Klaudia, et al.
Published: (2024)

Evaluating the Performance of Large Language Models via Debates
by: Moniri, Behrad, et al.
Published: (2024)

Rethinking Human Preference Evaluation of LLM Rationales
by: Li, Ziang, et al.
Published: (2025)

LoRe: Personalizing LLMs via Low-Rank Reward Modeling
by: Bose, Avinandan, et al.
Published: (2025)

Evaluating Gender Bias Transfer between Pre-trained and Prompt-Adapted Language Models
by: Mackraz, Natalie, et al.
Published: (2024)

Evaluation of Large Language Models via Coupled Token Generation
by: Benz, Nina Corvelo, et al.
Published: (2025)

Existing Large Language Model Unlearning Evaluations Are Inconclusive
by: Feng, Zhili, et al.
Published: (2025)

Anticipatory Evaluation of Language Models
by: Park, Jungsoo, et al.
Published: (2025)

Causal Evaluation of Language Models
by: Chen, Sirui, et al.
Published: (2024)

Toward the Evaluation of Large Language Models Considering Score Variance across Instruction Templates
by: Sakai, Yusuke, et al.
Published: (2024)

Omanic: Towards Step-wise Evaluation of Multi-hop Reasoning in Large Language Models
by: Gu, Xiaojie, et al.
Published: (2026)

MolMem: Memory-Augmented Agentic Reinforcement Learning for Sample-Efficient Molecular Optimization
by: Wang, Ziqing, et al.
Published: (2026)

Active Preference Learning for Large Language Models
by: Muldrew, William, et al.
Published: (2024)

Training and Evaluating Language Models with Template-based Data Generation
by: Zhang, Yifan
Published: (2024)

StructEval: Deepen and Broaden Large Language Model Assessment via Structured Evaluation
by: Cao, Boxi, et al.
Published: (2024)

Beyond the Singular: Revealing the Value of Multiple Generations in Benchmark Evaluation
by: Zhang, Wenbo, et al.
Published: (2025)

A Survey of Quantized Graph Representation Learning: Connecting Graph Structures with Large Language Models
by: Lin, Qika, et al.
Published: (2025)

Offline Training of Language Model Agents with Functions as Learnable Weights
by: Zhang, Shaokun, et al.
Published: (2024)

Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation
by: Cho, Jaemin, et al.
Published: (2023)

IntroLM: Introspective Language Models via Prefilling-Time Self-Evaluation
by: Kasnavieh, Hossein Hosseini, et al.
Published: (2026)

Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Results
by: Santilli, Andrea, et al.
Published: (2025)

Generate Any Scene: Scene Graph Driven Data Synthesis for Visual Generation Training
by: Gao, Ziqi, et al.
Published: (2024)

Beyond Bradley-Terry Models: A General Preference Model for Language Model Alignment
by: Zhang, Yifan, et al.
Published: (2024)

Report Cards: Qualitative Evaluation of Language Models Using Natural Language Summaries
by: Yang, Blair, et al.
Published: (2024)

Exploring Concept Depth: How Large Language Models Acquire Knowledge and Concept at Different Layers?
by: Jin, Mingyu, et al.
Published: (2024)