Imagen de Portada

A Unified Pairwise Framework for RLHF: Bridging Generative Reward Modeling and Policy Optimization

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Xu, Wenyuan, Zuo, Xiaochen, Xin, Chao, Yue, Yu, Yan, Lin, Wu, Yonghui
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Reward-Robust RLHF in LLMs
por: Yan, Yuzi, et al.
Publicado: (2024)

Policy Filtration for RLHF to Mitigate Noise in Reward Models
por: Zhang, Chuheng, et al.
Publicado: (2024)

Policy Optimization in RLHF: The Impact of Out-of-preference Data
por: Li, Ziniu, et al.
Publicado: (2023)

Factored Causal Representation Learning for Robust Reward Modeling in RLHF
por: Yang, Yupei, et al.
Publicado: (2026)

Learning a Pessimistic Reward Model in RLHF
por: Xu, Yinglun, et al.
Publicado: (2025)

PAG: Multi-Turn Reinforced LLM Self-Correction with Policy as Generative Verifier
por: Jiang, Yuhua, et al.
Publicado: (2025)

Optimal Design for Reward Modeling in RLHF
por: Scheid, Antoine, et al.
Publicado: (2024)

It Takes Two: On the Seamlessness between Reward and Policy Model in RLHF
por: Lu, Taiming, et al.
Publicado: (2024)

RLHF Workflow: From Reward Modeling to Online RLHF
por: Dong, Hanze, et al.
Publicado: (2024)

How to Evaluate Reward Models for RLHF
por: Frick, Evan, et al.
Publicado: (2024)

Unifying Stable Optimization and Reference Regularization in RLHF
por: He, Li, et al.
Publicado: (2026)

Reward Generalization in RLHF: A Topological Perspective
por: Qiu, Tianyi, et al.
Publicado: (2024)

Beyond RLHF: A Unified Theoretical Framework of Alignment
por: Yun, Jihun, et al.
Publicado: (2025)

Provably Efficient Online RLHF with One-Pass Reward Modeling
por: Li, Long-Fei, et al.
Publicado: (2025)

Can RLHF be More Efficient with Imperfect Reward Models? A Policy Coverage Perspective
por: Huang, Jiawei, et al.
Publicado: (2025)

Group Robust Preference Optimization in Reward-free RLHF
por: Ramesh, Shyam Sundhar, et al.
Publicado: (2024)

A Theoretical Framework for Partially Observed Reward-States in RLHF
por: Kausik, Chinmaya, et al.
Publicado: (2024)

Information-Theoretic Reward Modeling for Stable RLHF: Detecting and Mitigating Reward Hacking
por: Miao, Yuchun, et al.
Publicado: (2025)

On the Exponential Convergence for Offline RLHF with Pairwise Comparisons
por: Chen, Zhirui, et al.
Publicado: (2024)

Reward Model Overoptimisation in Iterated RLHF
por: Wolf, Lorenz, et al.
Publicado: (2025)

Dataset Reset Policy Optimization for RLHF
por: Chang, Jonathan D., et al.
Publicado: (2024)

Efficient Federated RLHF via Zeroth-Order Policy Optimization
por: Wang, Deyi, et al.
Publicado: (2026)

UFT: Unifying Fine-Tuning of SFT and RLHF/DPO/UNA through a Generalized Implicit Reward Function
por: Wang, Zhichao, et al.
Publicado: (2024)

Quantile Regression for Distributional Reward Models in RLHF
por: Dorka, Nicolai
Publicado: (2024)

Reward Shaping to Mitigate Reward Hacking in RLHF
por: Fu, Jiayi, et al.
Publicado: (2025)

Mitigating Reward Over-Optimization in RLHF via Behavior-Supported Regularization
por: Dai, Juntao, et al.
Publicado: (2025)

Robust Post-Training for Generative Recommenders: Why Exponential Reward-Weighted SFT Outperforms RLHF
por: Chidambaram, Keertana, et al.
Publicado: (2026)

Circuit-Aware Reward Training: A Mechanistic Framework for Longtail Robustness in RLHF
por: Liu, Jing
Publicado: (2025)

Information-Theoretic Reward Decomposition for Generalizable RLHF
por: Mao, Liyuan, et al.
Publicado: (2025)

One Framework to Rule Them All: Unifying RL-Based and RL-Free Methods in RLHF
por: Cai, Xin
Publicado: (2025)

Beyond Pairwise Preferences: Listwise Reward-Aware Alignment for Diffusion Models
por: Wang, Austin, et al.
Publicado: (2026)

Bias Fitting to Mitigate Length Bias of Reward Model in RLHF
por: Zhao, Kangwen, et al.
Publicado: (2025)

InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling
por: Miao, Yuchun, et al.
Publicado: (2024)

Projection Optimization: A General Framework for Multi-Objective and Multi-Group RLHF
por: Xiong, Nuoya, et al.
Publicado: (2025)

Policy Optimization Algorithms in a Unified Framework
por: Wu, Shuang
Publicado: (2025)

Towards a Theoretical Understanding to the Generalization of RLHF
por: Li, Zhaochun, et al.
Publicado: (2026)

BadReward: Clean-Label Poisoning of Reward Models in Text-to-Image RLHF
por: Duan, Kaiwen, et al.
Publicado: (2025)

OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework
por: Hu, Jian, et al.
Publicado: (2024)

Reward-aware Preference Optimization: A Unified Mathematical Framework for Model Alignment
por: Sun, Shengyang, et al.
Publicado: (2025)

PaTaRM: Bridging Pairwise and Pointwise Signals via Preference-Aware Task-Adaptive Reward Modeling
por: Jian, Ai, et al.
Publicado: (2025)