Image de couverture de livre

Circuit-Aware Reward Training: A Mechanistic Framework for Longtail Robustness in RLHF

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Liu, Jing
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Reward-Robust RLHF in LLMs
par: Yan, Yuzi, et autres
Publié: (2024)

Generative Data Mining with Longtail-Guided Diffusion
par: Hayden, David S., et autres
Publié: (2025)

A Theoretical Framework for Partially Observed Reward-States in RLHF
par: Kausik, Chinmaya, et autres
Publié: (2024)

Accelerating RLHF Training with Reward Variance Increase
par: Yang, Zonglin, et autres
Publié: (2025)

RLHF Workflow: From Reward Modeling to Online RLHF
par: Dong, Hanze, et autres
Publié: (2024)

Mitigating Reward Hacking in RLHF via Advantage Sign Robustness
par: Ono, Shinnosuke, et autres
Publié: (2026)

Reward Shaping to Mitigate Reward Hacking in RLHF
par: Fu, Jiayi, et autres
Publié: (2025)

Policy Filtration for RLHF to Mitigate Noise in Reward Models
par: Zhang, Chuheng, et autres
Publié: (2024)

Towards Reward Fairness in RLHF: From a Resource Allocation Perspective
par: Ouyang, Sheng, et autres
Publié: (2025)

Reward Model Overoptimisation in Iterated RLHF
par: Wolf, Lorenz, et autres
Publié: (2025)

How to Evaluate Reward Models for RLHF
par: Frick, Evan, et autres
Publié: (2024)

InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling
par: Miao, Yuchun, et autres
Publié: (2024)

Bias Fitting to Mitigate Length Bias of Reward Model in RLHF
par: Zhao, Kangwen, et autres
Publié: (2025)

Generalisation of RLHF under Reward Shift and Clipped KL Regularisation
par: Tang, Kenton, et autres
Publié: (2026)

Distributionally Robust Token Optimization in RLHF
par: Jin, Yeping, et autres
Publié: (2026)

An Adaptive Placement and Parallelism Framework for Accelerating RLHF Training
par: Xiao, Youshao, et autres
Publié: (2023)

Information-Theoretic Reward Decomposition for Generalizable RLHF
par: Mao, Liyuan, et autres
Publié: (2025)

Quantile Regression for Distributional Reward Models in RLHF
par: Dorka, Nicolai
Publié: (2024)

ODIN: Disentangled Reward Mitigates Hacking in RLHF
par: Chen, Lichang, et autres
Publié: (2024)

Mitigating Reward Over-Optimization in RLHF via Behavior-Supported Regularization
par: Dai, Juntao, et autres
Publié: (2025)

Can RLHF be More Efficient with Imperfect Reward Models? A Policy Coverage Perspective
par: Huang, Jiawei, et autres
Publié: (2025)

OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework
par: Hu, Jian, et autres
Publié: (2024)

BadReward: Clean-Label Poisoning of Reward Models in Text-to-Image RLHF
par: Duan, Kaiwen, et autres
Publié: (2025)

A Shared Low-Rank Adaptation Approach to Personalized RLHF
par: Liu, Renpu, et autres
Publié: (2025)

Reward Generalization in RLHF: A Topological Perspective
par: Qiu, Tianyi, et autres
Publié: (2024)

Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF
par: Zhu, Banghua, et autres
Publié: (2024)

It Takes Two: On the Seamlessness between Reward and Policy Model in RLHF
par: Lu, Taiming, et autres
Publié: (2024)

reward-lens: A Mechanistic Interpretability Library for Reward Models
par: Nadaf, Mohammed Suhail B
Publié: (2026)

RLHF in an SFT Way: From Optimal Solution to Reward-Weighted Alignment
par: Du, Yuhao, et autres
Publié: (2025)

CausalRM: Causal-Theoretic Reward Modeling for RLHF from Observational User Feedbacks
par: Wang, Hao, et autres
Publié: (2026)

A Unified Theoretical Analysis of Private and Robust Offline Alignment: from RLHF to DPO
par: Zhou, Xingyu, et autres
Publié: (2025)

Projection Optimization: A General Framework for Multi-Objective and Multi-Group RLHF
par: Xiong, Nuoya, et autres
Publié: (2025)

RewardUQ: A Unified Framework for Uncertainty-Aware Reward Models
par: Yang, Daniel, et autres
Publié: (2026)

Beyond RLHF and NLHF: Population-Proportional Alignment under an Axiomatic Framework
par: Kim, Kihyun, et autres
Publié: (2025)

Understanding Sampler Stochasticity in Training Diffusion Models for RLHF
par: Sheng, Jiayuan, et autres
Publié: (2025)

Greedy Sampling Is Provably Efficient for RLHF
par: Wu, Di, et autres
Publié: (2025)

ROCM: RLHF on consistency models
par: Shekhar, Shivanshu, et autres
Publié: (2025)

G-Core: A Simple, Scalable and Balanced RLHF Trainer
par: Wu, Junyu, et autres
Publié: (2025)

The Hidden Link Between RLHF and Contrastive Learning
par: Lv, Xufei, et autres
Publié: (2025)

Rethinking KL Regularization in RLHF: From Value Estimation to Gradient Optimization
par: Liu, Kezhao, et autres
Publié: (2025)