Copertina

It Takes Two: On the Seamlessness between Reward and Policy Model in RLHF

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Lu, Taiming, Shen, Lingfeng, Yang, Xinyu, Tan, Weiting, Chen, Beidi, Yao, Huaxiu
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

RLHF Workflow: From Reward Modeling to Online RLHF
di: Dong, Hanze, et al.
Pubblicazione: (2024)

Reward Shaping to Mitigate Reward Hacking in RLHF
di: Fu, Jiayi, et al.
Pubblicazione: (2025)

Reward-Robust RLHF in LLMs
di: Yan, Yuzi, et al.
Pubblicazione: (2024)

How to Evaluate Reward Models for RLHF
di: Frick, Evan, et al.
Pubblicazione: (2024)

Reward Model Overoptimisation in Iterated RLHF
di: Wolf, Lorenz, et al.
Pubblicazione: (2025)

Quantile Regression for Distributional Reward Models in RLHF
di: Dorka, Nicolai
Pubblicazione: (2024)

ODIN: Disentangled Reward Mitigates Hacking in RLHF
di: Chen, Lichang, et al.
Pubblicazione: (2024)

Policy Filtration for RLHF to Mitigate Noise in Reward Models
di: Zhang, Chuheng, et al.
Pubblicazione: (2024)

CausalRM: Causal-Theoretic Reward Modeling for RLHF from Observational User Feedbacks
di: Wang, Hao, et al.
Pubblicazione: (2026)

Information-Theoretic Reward Decomposition for Generalizable RLHF
di: Mao, Liyuan, et al.
Pubblicazione: (2025)

Reward Generalization in RLHF: A Topological Perspective
di: Qiu, Tianyi, et al.
Pubblicazione: (2024)

RLHF in an SFT Way: From Optimal Solution to Reward-Weighted Alignment
di: Du, Yuhao, et al.
Pubblicazione: (2025)

Dataset Reset Policy Optimization for RLHF
di: Chang, Jonathan D., et al.
Pubblicazione: (2024)

Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF
di: Zhu, Banghua, et al.
Pubblicazione: (2024)

Mitigating Reward Hacking in RLHF via Advantage Sign Robustness
di: Ono, Shinnosuke, et al.
Pubblicazione: (2026)

Ever: Mitigating Hallucination in Large Language Models through Real-Time Verification and Rectification
di: Kang, Haoqiang, et al.
Pubblicazione: (2023)

OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework
di: Hu, Jian, et al.
Pubblicazione: (2024)

Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models
di: Noukhovitch, Michael, et al.
Pubblicazione: (2024)

Cost-Effective Proxy Reward Model Construction with On-Policy and Active Learning
di: Chen, Yifang, et al.
Pubblicazione: (2024)

S$^{2}$FT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsity
di: Yang, Xinyu, et al.
Pubblicazione: (2024)

RLHF and IIA: Perverse Incentives
di: Xu, Wanqiao, et al.
Pubblicazione: (2023)

Regressing the Relative Future: Efficient Policy Optimization for Multi-turn RLHF
di: Gao, Zhaolin, et al.
Pubblicazione: (2024)

APE: Faster and Longer Context-Augmented Generation via Adaptive Parallel Encoding
di: Yang, Xinyu, et al.
Pubblicazione: (2025)

Prompt-prompted Adaptive Structured Pruning for Efficient LLM Generation
di: Dong, Harry, et al.
Pubblicazione: (2024)

Exploration-Driven Policy Optimization in RLHF: Theoretical Insights on Efficient Data Utilization
di: Du, Yihan, et al.
Pubblicazione: (2024)

Towards Data-Centric RLHF: Simple Metrics for Preference Dataset Comparison
di: Shen, Judy Hanwen, et al.
Pubblicazione: (2024)

Co-Evolution of Policy and Internal Reward for Language Agents
di: Wang, Xinyu, et al.
Pubblicazione: (2026)

BadReward: Clean-Label Poisoning of Reward Models in Text-to-Image RLHF
di: Duan, Kaiwen, et al.
Pubblicazione: (2025)

Can RLHF be More Efficient with Imperfect Reward Models? A Policy Coverage Perspective
di: Huang, Jiawei, et al.
Pubblicazione: (2025)

Proxy-RLHF: Decoupling Generation and Alignment in Large Language Model with Proxy
di: Zhu, Yu, et al.
Pubblicazione: (2024)

ReDit: Reward Dithering for Improved LLM Policy Optimization
di: Wei, Chenxing, et al.
Pubblicazione: (2025)

SALMON: Self-Alignment with Instructable Reward Models
di: Sun, Zhiqing, et al.
Pubblicazione: (2023)

Fine-Tuning Language Models with Reward Learning on Policy
di: Lang, Hao, et al.
Pubblicazione: (2024)

Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation
di: Yang, Wenkai, et al.
Pubblicazione: (2026)

Rethinking Reward Model Evaluation: Are We Barking up the Wrong Tree?
di: Wen, Xueru, et al.
Pubblicazione: (2024)

Scalable LLM Reasoning Acceleration with Low-rank Distillation
di: Dong, Harry, et al.
Pubblicazione: (2025)

Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Data
di: Ling, Zhenqing, et al.
Pubblicazione: (2025)

FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guarantees
di: Nie, Fan, et al.
Pubblicazione: (2024)

The Policy Cliff: A Theoretical Analysis of Reward-Policy Maps in Large Language Models
di: Xu, Xingcheng
Pubblicazione: (2025)

The Perfect Blend: Redefining RLHF with Mixture of Judges
di: Xu, Tengyu, et al.
Pubblicazione: (2024)