Copertina

The Flip Side of RLHF: On-Policy Feedback for Reward Model Self-Supervised Improvement

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Xiaobo, Wu, Tong, Tang, Min, Li, Jiaqi, Liu, Qi, Zheng, Zilong
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedback
di: Zhou, Jiayi, et al.
Pubblicazione: (2024)

Adaptive Preference Optimization with Uncertainty-aware Utility Anchor
di: Wang, Xiaobo, et al.
Pubblicazione: (2025)

CausalRM: Causal-Theoretic Reward Modeling for RLHF from Observational User Feedbacks
di: Wang, Hao, et al.
Pubblicazione: (2026)

RLHF Workflow: From Reward Modeling to Online RLHF
di: Dong, Hanze, et al.
Pubblicazione: (2024)

In-Context Editing: Learning Knowledge from Self-Induced Distributions
di: Qi, Siyuan, et al.
Pubblicazione: (2024)

Reward Shaping to Mitigate Reward Hacking in RLHF
di: Fu, Jiayi, et al.
Pubblicazione: (2025)

ChatGLM-RLHF: Practices of Aligning Large Language Models with Human Feedback
di: Hou, Zhenyu, et al.
Pubblicazione: (2024)

It Takes Two: On the Seamlessness between Reward and Policy Model in RLHF
di: Lu, Taiming, et al.
Pubblicazione: (2024)

How to Evaluate Reward Models for RLHF
di: Frick, Evan, et al.
Pubblicazione: (2024)

Reward-Robust RLHF in LLMs
di: Yan, Yuzi, et al.
Pubblicazione: (2024)

Prototypical Reward Network for Data-Efficient RLHF
di: Zhang, Jinghan, et al.
Pubblicazione: (2024)

Reward Model Overoptimisation in Iterated RLHF
di: Wolf, Lorenz, et al.
Pubblicazione: (2025)

Taming Overconfidence in LLMs: Reward Calibration in RLHF
di: Leng, Jixuan, et al.
Pubblicazione: (2024)

MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions
di: Chai, Yekun, et al.
Pubblicazione: (2024)

Quantile Regression for Distributional Reward Models in RLHF
di: Dorka, Nicolai
Pubblicazione: (2024)

LongRM: Revealing and Unlocking the Context Boundary of Reward Modeling
di: Tang, Zecheng, et al.
Pubblicazione: (2025)

ARF-RLHF: Adaptive Reward-Following for RLHF through Emotion-Driven Self-Supervision and Trace-Biased Dynamic Optimization
di: Zhang, YuXuan
Pubblicazione: (2025)

Policy Improvement using Language Feedback Models
di: Zhong, Victor, et al.
Pubblicazione: (2024)

Reward Modeling from Natural Language Human Feedback
di: Wang, Zongqi, et al.
Pubblicazione: (2026)

Segmenting Text and Learning Their Rewards for Improved RLHF in Language Model
di: Yin, Yueqin, et al.
Pubblicazione: (2025)

Reward Difference Optimization For Sample Reweighting In Offline RLHF
di: Wang, Shiqi, et al.
Pubblicazione: (2024)

LooGLE: Can Long-Context Language Models Understand Long Contexts?
di: Li, Jiaqi, et al.
Pubblicazione: (2023)

RAM: Towards an Ever-Improving Memory System by Learning from Communications
di: Li, Jiaqi, et al.
Pubblicazione: (2024)

An Efficient Recipe for Long Context Extension via Middle-Focused Positional Encoding
di: Wu, Tong, et al.
Pubblicazione: (2024)

RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic Sampling
di: Liu, Yang, et al.
Pubblicazione: (2025)

SCAN: Self-Denoising Monte Carlo Annotation for Robust Process Reward Learning
di: Ding, Yuyang, et al.
Pubblicazione: (2025)

Evaluating Defences against Unsafe Feedback in RLHF
di: Rosati, Domenic, et al.
Pubblicazione: (2024)

Group Robust Preference Optimization in Reward-free RLHF
di: Ramesh, Shyam Sundhar, et al.
Pubblicazione: (2024)

Continual SFT Matches Multimodal RLHF with Negative Supervision
di: Zhu, Ke, et al.
Pubblicazione: (2024)

ODIN: Disentangled Reward Mitigates Hacking in RLHF
di: Chen, Lichang, et al.
Pubblicazione: (2024)

Information-Theoretic Reward Decomposition for Generalizable RLHF
di: Mao, Liyuan, et al.
Pubblicazione: (2025)

Test-time Recursive Thinking: Self-Improvement without External Feedback
di: Zhuang, Yufan, et al.
Pubblicazione: (2026)

Reward Generalization in RLHF: A Topological Perspective
di: Qiu, Tianyi, et al.
Pubblicazione: (2024)

Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision
di: He, Yinghui, et al.
Pubblicazione: (2026)

Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoning
di: Xu, Huimin, et al.
Pubblicazione: (2025)

Better Process Supervision with Bi-directional Rewarding Signals
di: Chen, Wenxiang, et al.
Pubblicazione: (2025)

Hindsight-Anchored Policy Optimization: Turning Failure into Feedback in Sparse Reward Settings
di: Wu, Yuning, et al.
Pubblicazione: (2026)

Enabling Language Models to Implicitly Learn Self-Improvement
di: Wang, Ziqi, et al.
Pubblicazione: (2023)

The Accuracy Paradox in RLHF: When Better Reward Models Don't Yield Better Language Models
di: Chen, Yanjun, et al.
Pubblicazione: (2024)

Progress or Regress? Self-Improvement Reversal in Post-training
di: Wu, Ting, et al.
Pubblicazione: (2024)