Cover Image

OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hu, Jian, Wu, Xibin, Shen, Wei, Liu, Jason Klein, Zhu, Zilin, Wang, Weixun, Jiang, Songlin, Wang, Haoran, Chen, Hao, Chen, Bin, Fang, Weikai, Xianyu, Cao, Yu, Xu, Haotian, Liu, Yiming
Format:	Preprint
Published:	2024
Subjects:	Artificial Intelligence Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Rethinking KL Regularization in RLHF: From Value Estimation to Gradient Optimization
by: Liu, Kezhao, et al.
Published: (2025)

Reward-Robust RLHF in LLMs
by: Yan, Yuzi, et al.
Published: (2024)

HybridFlow: A Flexible and Efficient RLHF Framework
by: Sheng, Guangming, et al.
Published: (2024)

The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarization
by: Huang, Shengyi, et al.
Published: (2024)

RLHF Workflow: From Reward Modeling to Online RLHF
by: Dong, Hanze, et al.
Published: (2024)

Prototypical Reward Network for Data-Efficient RLHF
by: Zhang, Jinghan, et al.
Published: (2024)

MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions
by: Chai, Yekun, et al.
Published: (2024)

G-Core: A Simple, Scalable and Balanced RLHF Trainer
by: Wu, Junyu, et al.
Published: (2025)

Continual SFT Matches Multimodal RLHF with Negative Supervision
by: Zhu, Ke, et al.
Published: (2024)

A Shared Low-Rank Adaptation Approach to Personalized RLHF
by: Liu, Renpu, et al.
Published: (2025)

The Hidden Link Between RLHF and Contrastive Learning
by: Lv, Xufei, et al.
Published: (2025)

Boosting Deductive Reasoning with Step Signals In RLHF
by: Li, Jialian, et al.
Published: (2024)

RLHFless: Serverless Computing for Efficient RLHF
by: Wei, Rui, et al.
Published: (2026)

Information-Theoretic Reward Decomposition for Generalizable RLHF
by: Mao, Liyuan, et al.
Published: (2025)

Towards a Theoretical Understanding to the Generalization of RLHF
by: Li, Zhaochun, et al.
Published: (2026)

Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHF
by: Shen, Han, et al.
Published: (2024)

Mitigating the Alignment Tax of RLHF
by: Lin, Yong, et al.
Published: (2023)

Towards Federated RLHF with Aggregated Client Preference for LLMs
by: Wu, Feijie, et al.
Published: (2024)

Greedy Sampling Is Provably Efficient for RLHF
by: Wu, Di, et al.
Published: (2025)

Dataset Reset Policy Optimization for RLHF
by: Chang, Jonathan D., et al.
Published: (2024)

Unifying Stable Optimization and Reference Regularization in RLHF
by: He, Li, et al.
Published: (2026)

Investigating on RLHF methodology
by: Kutalev, Alexey, et al.
Published: (2024)

SharedRep-RLHF: A Shared Representation Approach to RLHF with Diverse Preferences
by: Mukherjee, Arpan, et al.
Published: (2025)

ODIN: Disentangled Reward Mitigates Hacking in RLHF
by: Chen, Lichang, et al.
Published: (2024)

On the Exponential Convergence for Offline RLHF with Pairwise Comparisons
by: Chen, Zhirui, et al.
Published: (2024)

The Perfect Blend: Redefining RLHF with Mixture of Judges
by: Xu, Tengyu, et al.
Published: (2024)

FedRLHF: A Convergence-Guaranteed Federated Framework for Privacy-Preserving and Personalized RLHF
by: Fan, Flint Xiaofeng, et al.
Published: (2024)

It Takes Two: On the Seamlessness between Reward and Policy Model in RLHF
by: Lu, Taiming, et al.
Published: (2024)

How to Evaluate Reward Models for RLHF
by: Frick, Evan, et al.
Published: (2024)

On a Connection Between Imitation Learning and RLHF
by: Xiao, Teng, et al.
Published: (2025)

ROCM: RLHF on consistency models
by: Shekhar, Shivanshu, et al.
Published: (2025)

How RLHF Amplifies Sycophancy
by: Shapira, Itai, et al.
Published: (2026)

RLHF and IIA: Perverse Incentives
by: Xu, Wanqiao, et al.
Published: (2023)

Understanding and Alleviating Memory Consumption in RLHF for LLMs
by: Zhou, Jin, et al.
Published: (2024)

RLHS: Mitigating Misalignment in RLHF with Hindsight Simulation
by: Liang, Kaiqu, et al.
Published: (2025)

When Policy Entropy Constraint Fails: Preserving Diversity in Flow-based RLHF via Perceptual Entropy
by: Tan, Xiaofeng, et al.
Published: (2026)

Secrets of RLHF in Large Language Models Part II: Reward Modeling
by: Wang, Binghai, et al.
Published: (2024)

Reward Shaping to Mitigate Reward Hacking in RLHF
by: Fu, Jiayi, et al.
Published: (2025)

SCAR: Shapley Credit Assignment for More Efficient RLHF
by: Cao, Meng, et al.
Published: (2025)

Towards Reward Fairness in RLHF: From a Resource Allocation Perspective
by: Ouyang, Sheng, et al.
Published: (2025)