Copertina

RLHF Workflow: From Reward Modeling to Online RLHF

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Dong, Hanze, Xiong, Wei, Pang, Bo, Wang, Haoxiang, Zhao, Han, Zhou, Yingbo, Jiang, Nan, Sahoo, Doyen, Xiong, Caiming, Zhang, Tong
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Artificial Intelligence Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Automatic Curriculum Expert Iteration for Reliable LLM Reasoning
di: Zhao, Zirui, et al.
Pubblicazione: (2024)

A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce
di: Xiong, Wei, et al.
Pubblicazione: (2025)

Reward-Guided Speculative Decoding for Efficient LLM Reasoning
di: Liao, Baohao, et al.
Pubblicazione: (2025)

INDICT: Code Generation with Internal Dialogues of Critiques for Both Security and Helpfulness
di: Le, Hung, et al.
Pubblicazione: (2024)

Scalable Chain of Thoughts via Elastic Reasoning
di: Xu, Yuhui, et al.
Pubblicazione: (2025)

BOLT: Bootstrap Long Chain-of-Thought in Language Models without Distillation
di: Pang, Bo, et al.
Pubblicazione: (2025)

Reward Models Identify Consistency, Not Causality
di: Xu, Yuhui, et al.
Pubblicazione: (2025)

Fractured Chain-of-Thought Reasoning
di: Liao, Baohao, et al.
Pubblicazione: (2025)

Reward Shaping to Mitigate Reward Hacking in RLHF
di: Fu, Jiayi, et al.
Pubblicazione: (2025)

Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedback
di: Zhou, Jiayi, et al.
Pubblicazione: (2024)

Reward-Robust RLHF in LLMs
di: Yan, Yuzi, et al.
Pubblicazione: (2024)

How to Evaluate Reward Models for RLHF
di: Frick, Evan, et al.
Pubblicazione: (2024)

Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraint
di: Xiong, Wei, et al.
Pubblicazione: (2023)

Reward Model Overoptimisation in Iterated RLHF
di: Wolf, Lorenz, et al.
Pubblicazione: (2025)

Reward Generalization in RLHF: A Topological Perspective
di: Qiu, Tianyi, et al.
Pubblicazione: (2024)

ThinK: Thinner Key Cache by Query-Driven Pruning
di: Xu, Yuhui, et al.
Pubblicazione: (2024)

Reasoning Curriculum: Bootstrapping Broad LLM Reasoning from Math
di: Pang, Bo, et al.
Pubblicazione: (2025)

DPO Meets PPO: Reinforced Token Optimization for RLHF
di: Zhong, Han, et al.
Pubblicazione: (2024)

CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models
di: Li, Jierui, et al.
Pubblicazione: (2024)

Parameter-Efficient Detoxification with Contrastive Decoding
di: Niu, Tong, et al.
Pubblicazione: (2024)

Segmenting Text and Learning Their Rewards for Improved RLHF in Language Model
di: Yin, Yueqin, et al.
Pubblicazione: (2025)

Quantile Regression for Distributional Reward Models in RLHF
di: Dorka, Nicolai
Pubblicazione: (2024)

Reward Difference Optimization For Sample Reweighting In Offline RLHF
di: Wang, Shiqi, et al.
Pubblicazione: (2024)

ODIN: Disentangled Reward Mitigates Hacking in RLHF
di: Chen, Lichang, et al.
Pubblicazione: (2024)

Prototypical Reward Network for Data-Efficient RLHF
di: Zhang, Jinghan, et al.
Pubblicazione: (2024)

Information-Theoretic Reward Decomposition for Generalizable RLHF
di: Mao, Liyuan, et al.
Pubblicazione: (2025)

OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework
di: Hu, Jian, et al.
Pubblicazione: (2024)

Proxy-RLHF: Decoupling Generation and Alignment in Large Language Model with Proxy
di: Zhu, Yu, et al.
Pubblicazione: (2024)

RLHF in an SFT Way: From Optimal Solution to Reward-Weighted Alignment
di: Du, Yuhao, et al.
Pubblicazione: (2025)

It Takes Two: On the Seamlessness between Reward and Policy Model in RLHF
di: Lu, Taiming, et al.
Pubblicazione: (2024)

The Accuracy Paradox in RLHF: When Better Reward Models Don't Yield Better Language Models
di: Chen, Yanjun, et al.
Pubblicazione: (2024)

JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking
di: Niu, Tong, et al.
Pubblicazione: (2024)

Mitigating the Alignment Tax of RLHF
di: Lin, Yong, et al.
Pubblicazione: (2023)

PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedback
di: Peng, Yun, et al.
Pubblicazione: (2024)

RLHF and IIA: Perverse Incentives
di: Xu, Wanqiao, et al.
Pubblicazione: (2023)

Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF
di: Zhu, Banghua, et al.
Pubblicazione: (2024)

Mitigating Reward Hacking in RLHF via Advantage Sign Robustness
di: Ono, Shinnosuke, et al.
Pubblicazione: (2026)

Policy Filtration for RLHF to Mitigate Noise in Reward Models
di: Zhang, Chuheng, et al.
Pubblicazione: (2024)

Entropy-Based Block Pruning for Efficient Large Language Models
di: Yang, Liangwei, et al.
Pubblicazione: (2025)

CausalRM: Causal-Theoretic Reward Modeling for RLHF from Observational User Feedbacks
di: Wang, Hao, et al.
Pubblicazione: (2026)