Cover Image

Explicit Preference Optimization: No Need for an Implicit Reward Model

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hu, Xiangkun, Kong, Lemin, He, Tong, Wipf, David
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

New Desiderata for Direct Preference Optimization
by: Hu, Xiangkun, et al.
Published: (2024)

On the Limited Generalization Capability of the Implicit Reward Model Induced by Direct Preference Optimization
by: Lin, Yong, et al.
Published: (2024)

Auto-Rubric: Learning From Implicit Weights to Explicit Rubrics for Reward Modeling
by: Xie, Lipeng, et al.
Published: (2025)

SPO: Multi-Dimensional Preference Sequential Alignment With Implicit Reward Modeling
by: Lou, Xingzhou, et al.
Published: (2024)

BloomGML: Graph Machine Learning through the Lens of Bilevel Optimization
by: Zheng, Amber Yijia, et al.
Published: (2024)

Implicit vs Unfolded Graph Neural Networks
by: Yang, Yongyi, et al.
Published: (2021)

From Demonstrations to Rewards: Alignment Without Explicit Human Preferences
by: Zeng, Siliang, et al.
Published: (2025)

Explicit and Implicit Graduated Optimization in Deep Neural Networks
by: Sato, Naoki, et al.
Published: (2024)

No Need to Train Your RDB Foundation Model
by: Xu, Linjie, et al.
Published: (2026)

Sparse Autoencoders, Again?
by: Lu, Yin, et al.
Published: (2025)

Noise Contrastive Alignment of Language Models with Explicit Rewards
by: Chen, Huayu, et al.
Published: (2024)

Robust Preference Optimization through Reward Model Distillation
by: Fisch, Adam, et al.
Published: (2024)

Preference as Reward, Maximum Preference Optimization with Importance Sampling
by: Jiang, Zaifan, et al.
Published: (2023)

Group Robust Preference Optimization in Reward-free RLHF
by: Ramesh, Shyam Sundhar, et al.
Published: (2024)

AMoPO: Adaptive Multi-objective Preference Optimization without Reward Models and Reference Models
by: Liu, Qi, et al.
Published: (2025)

PREFINE: Preference-Based Implicit Reward and Cost Fine-Tuning for Safety Alignment
by: Verma, Richa, et al.
Published: (2026)

Multi-Objective Reward and Preference Optimization: Theory and Algorithms
by: Agnihotri, Akhil
Published: (2025)

Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap
by: Qi, Xuan, et al.
Published: (2025)

SLIME: Stabilized Likelihood Implicit Margin Enforcement for Preference Optimization
by: Afanasyev, Maksim, et al.
Published: (2026)

Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts
by: Wang, Haoxiang, et al.
Published: (2024)

Reward-aware Preference Optimization: A Unified Mathematical Framework for Model Alignment
by: Sun, Shengyang, et al.
Published: (2025)

Binary Reward Labeling: Bridging Offline Preference and Reward-Based Reinforcement Learning
by: Xu, Yinglun, et al.
Published: (2024)

InfiFPO: Implicit Model Fusion via Preference Optimization in Large Language Models
by: Gu, Yanggan, et al.
Published: (2025)

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
by: Wu, Junkang, et al.
Published: (2024)

Optimal Transport for LLM Reward Modeling from Noisy Preference
by: Pan, Licheng, et al.
Published: (2026)

Bootstrapping Language Models with DPO Implicit Rewards
by: Chen, Changyu, et al.
Published: (2024)

Alignment with Preference Optimization Is All You Need for LLM Safety
by: Alami, Reda, et al.
Published: (2024)

Margin Adaptive DPO: Leveraging Reward Model for Granular Control in Preference Optimization
by: Rho, Hyung Gyu
Published: (2025)

PILAF: Optimal Human Preference Sampling for Reward Modeling
by: Feng, Yunzhen, et al.
Published: (2025)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
by: Rafailov, Rafael, et al.
Published: (2023)

Reward-Zero: Language Embedding Driven Implicit Reward Mechanisms for Reinforcement Learning
by: Zhang, Heng, et al.
Published: (2026)

Robust Angular Synchronization via Directed Graph Neural Networks
by: He, Yixuan, et al.
Published: (2023)

Dual-Agent Co-Training for Health Coaching via Implicit Adversarial Preference Optimization
by: Long, Da, et al.
Published: (2026)

A Comprehensive Study of Implicit and Explicit Biases in Large Language Models
by: Kazi, Fatima, et al.
Published: (2025)

In-Context Reward Adaptation for Robust Preference Modeling
by: Sun, Zhenyu, et al.
Published: (2026)

T-REG: Preference Optimization with Token-Level Reward Regularization
by: Zhou, Wenxuan, et al.
Published: (2024)

Reward Model Learning vs. Direct Policy Optimization: A Comparative Analysis of Learning from Human Preferences
by: Nika, Andi, et al.
Published: (2024)

Pragmatic Feature Preferences: Learning Reward-Relevant Preferences from Human Input
by: Peng, Andi, et al.
Published: (2024)

On the Implicit Reward Overfitting and the Low-rank Dynamics in RLVR
by: Ye, Hao, et al.
Published: (2026)

A Unified Pair-GRPO Family: From Implicit to Explicit Preference Constraints for Stable and General RL Alignment
by: Yu, Hao
Published: (2026)