Cover Image

Rethinking Bradley-Terry Models in Preference-Based Reward Modeling: Foundations, Theory, and Alternatives

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Sun, Hao, Shen, Yunyi, Ton, Jean-Francois
Format:	Preprint
Published:	2024
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Reviving The Classics: Active Reward Modeling in Large Language Model Alignment
by: Shen, Yunyi, et al.
Published: (2025)

Reusing Embeddings: Reproducible Reward Model Research in Large Language Model Alignment without GPUs
by: Sun, Hao, et al.
Published: (2025)

Beyond Bradley-Terry Models: A General Preference Model for Language Model Alignment
by: Zhang, Yifan, et al.
Published: (2024)

Efficient Portfolio Selection through Preference Aggregation with Quicksort and the Bradley--Terry Model
by: Ge, Yurun, et al.
Published: (2025)

OpenDeepThink: Parallel Reasoning via Bradley-Terry Aggregation
by: Zhou, Shang, et al.
Published: (2026)

Neural Bradley-Terry Rating: Quantifying Properties from Comparisons
by: Fujii, Satoru
Published: (2023)

Overcoming Reward Overoptimization via Adversarial Policy Optimization with Lightweight Uncertainty Estimation
by: Zhang, Xiaoying, et al.
Published: (2024)

Bradley-Terry and Multi-Objective Reward Modeling Are Complementary
by: Zhang, Zhiwei, et al.
Published: (2025)

OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Models
by: Sun, Hao, et al.
Published: (2025)

Bradley-Terry Policy Optimization for Generative Preference Modeling
by: Feng, Shengyu, et al.
Published: (2025)

Understanding Chain-of-Thought in LLMs through Information Theory
by: Ton, Jean-Francois, et al.
Published: (2024)

In-Context Reward Adaptation for Robust Preference Modeling
by: Sun, Zhenyu, et al.
Published: (2026)

Optimal Transport for LLM Reward Modeling from Noisy Preference
by: Pan, Licheng, et al.
Published: (2026)

Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment
by: Yang, Rui, et al.
Published: (2024)

Energy-Based Preference Model Offers Better Offline Alignment than the Bradley-Terry Preference Model
by: Hong, Yuzhong, et al.
Published: (2024)

Towards Comprehensive Preference Data Collection for Reward Modeling
by: Hu, Yulan, et al.
Published: (2024)

Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems
by: Peng, Hao, et al.
Published: (2025)

Rethinking Reward Models for Multi-Domain Test-Time Scaling
by: Lee, Dong Bok, et al.
Published: (2025)

Rethinking Diverse Human Preference Learning through Principal Component Analysis
by: Luo, Feng, et al.
Published: (2025)

Rethinking Reward Model Evaluation Through the Lens of Reward Overoptimization
by: Kim, Sunghwan, et al.
Published: (2025)

Aligning Crowd Feedback via Distributional Preference Reward Modeling
by: Li, Dexun, et al.
Published: (2024)

GRAM: A Generative Foundation Reward Model for Reward Generalization
by: Wang, Chenglong, et al.
Published: (2025)

ImplicitRM: Unbiased Reward Modeling from Implicit Preference Data for LLM alignment
by: Wang, Hao, et al.
Published: (2026)

The Bradley-Terry Stochastic Block Model
by: Santi, Lapo, et al.
Published: (2025)

AMoPO: Adaptive Multi-objective Preference Optimization without Reward Models and Reference Models
by: Liu, Qi, et al.
Published: (2025)

IRPM: Intergroup Relative Preference Modeling for Pointwise Generative Reward Models
by: Song, Haonan, et al.
Published: (2026)

Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Tasks
by: Shen, William F., et al.
Published: (2026)

Provable Reward-Agnostic Preference-Based Reinforcement Learning
by: Zhan, Wenhao, et al.
Published: (2023)

Preference Poisoning Attacks on Reward Model Learning
by: Wu, Junlin, et al.
Published: (2024)

Residual Reward Models for Preference-based Reinforcement Learning
by: Cao, Chenyang, et al.
Published: (2025)

PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Model
by: Lin, Baijiong, et al.
Published: (2025)

Rethinking Reward Model Evaluation: Are We Barking up the Wrong Tree?
by: Wen, Xueru, et al.
Published: (2024)

ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaboration
by: Estornell, Andrew, et al.
Published: (2024)

Rethinking the Role of Proxy Rewards in Language Model Alignment
by: Kim, Sungdong, et al.
Published: (2024)

Towards Understanding the Influence of Reward Margin on Preference Model Performance
by: Qin, Bowen, et al.
Published: (2024)

Structural Reward Model: Enhancing Interpretability, Efficiency, and Scalability in Reward Modeling
by: Liu, Xiaoyu, et al.
Published: (2025)

Beyond Scalar Reward Model: Learning Generative Judge from Preference Data
by: Ye, Ziyi, et al.
Published: (2024)

The Bayesian Intransitive Bradley-Terry Model via Combinatorial Hodge Theory
by: Okahara, Hisaya, et al.
Published: (2026)

Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward
by: Zhang, Ruohong, et al.
Published: (2024)

A Foundation Model for Massive MIMO Precoding with an Adaptive per-User Rate-Power Tradeoff
by: Emery, Jérôme, et al.
Published: (2025)