Image de couverture de livre

Robust Post-Training for Generative Recommenders: Why Exponential Reward-Weighted SFT Outperforms RLHF

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Chidambaram, Keertana, Krishnamurthy, Sanath Kumar, Xu, Qiuling, Hsiao, Ko-Jen, Bhattacharya, Moumita
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Towards Generalizable and Efficient Large-Scale Generative Recommenders
par: Xu, Qiuling, et autres
Publié: (2026)

RLHF in an SFT Way: From Optimal Solution to Reward-Weighted Alignment
par: Du, Yuhao, et autres
Publié: (2025)

Direct Preference Optimization With Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences
par: Chidambaram, Keertana, et autres
Publié: (2024)

UFT: Unifying Fine-Tuning of SFT and RLHF/DPO/UNA through a Generalized Implicit Reward Function
par: Wang, Zhichao, et autres
Publié: (2024)

Sliding Window Training -- Utilizing Historical Recommender Systems Data for Foundation Models
par: Joshi, Swanand, et autres
Publié: (2024)

Reward-Robust RLHF in LLMs
par: Yan, Yuzi, et autres
Publié: (2024)

Joint Modeling of Search and Recommendations Via an Unified Contextual Recommender (UniCoRn)
par: Bhattacharya, Moumita, et autres
Publié: (2024)

Why Does RL Generalize Better Than SFT? A Data-Centric Perspective on VLM Post-Training
par: Lu, Aojun, et autres
Publié: (2026)

Sequential Decision Making with Expert Demonstrations under Unobserved Heterogeneity
par: Balazadeh, Vahid, et autres
Publié: (2024)

Circuit-Aware Reward Training: A Mechanistic Framework for Longtail Robustness in RLHF
par: Liu, Jing
Publié: (2025)

Factored Causal Representation Learning for Robust Reward Modeling in RLHF
par: Yang, Yupei, et autres
Publié: (2026)

Continual SFT Matches Multimodal RLHF with Negative Supervision
par: Zhu, Ke, et autres
Publié: (2024)

Accelerating RLHF Training with Reward Variance Increase
par: Yang, Zonglin, et autres
Publié: (2025)

On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification
par: Wu, Yongliang, et autres
Publié: (2025)

SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
par: Limozin, Alexis, et autres
Publié: (2026)

Group Robust Preference Optimization in Reward-free RLHF
par: Ramesh, Shyam Sundhar, et autres
Publié: (2024)

Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer
par: Liu, Zhihan, et autres
Publié: (2024)

Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
par: Kang, Feiyang, et autres
Publié: (2025)

Learning a Pessimistic Reward Model in RLHF
par: Xu, Yinglun, et autres
Publié: (2025)

RLHF Workflow: From Reward Modeling to Online RLHF
par: Dong, Hanze, et autres
Publié: (2024)

Personalized Adaptation via In-Context Preference Learning
par: Lau, Allison, et autres
Publié: (2024)

Mitigating Reward Hacking in RLHF via Advantage Sign Robustness
par: Ono, Shinnosuke, et autres
Publié: (2026)

RLSR: Reinforcement Learning with Supervised Reward Outperforms SFT in Instruction Following
par: Wang, Zhichao, et autres
Publié: (2025)

Reward Generalization in RLHF: A Topological Perspective
par: Qiu, Tianyi, et autres
Publié: (2024)

Optimal Design for Reward Modeling in RLHF
par: Scheid, Antoine, et autres
Publié: (2024)

Direct Preference Optimization with Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences
par: Chidambaram, Keertana, et autres
Publié: (2025)

Reward Shaping to Mitigate Reward Hacking in RLHF
par: Fu, Jiayi, et autres
Publié: (2025)

Information-Theoretic Reward Decomposition for Generalizable RLHF
par: Mao, Liyuan, et autres
Publié: (2025)

SFT-GRPO Data Overlap as a Post-Training Hyperparameter for Autoformalization
par: Su, Xiaole, et autres
Publié: (2026)

GAC: Noise-Aware Adaptive Mixing for Hybrid SFT-RL Post-Training
par: Hu, Yuelin, et autres
Publié: (2026)

A Unified Pairwise Framework for RLHF: Bridging Generative Reward Modeling and Policy Optimization
par: Xu, Wenyuan, et autres
Publié: (2025)

Policy Filtration for RLHF to Mitigate Noise in Reward Models
par: Zhang, Chuheng, et autres
Publié: (2024)

Reward Model Overoptimisation in Iterated RLHF
par: Wolf, Lorenz, et autres
Publié: (2025)

How to Evaluate Reward Models for RLHF
par: Frick, Evan, et autres
Publié: (2024)

Robust inverse material design with physical guarantees using the Voigt-Reuss Net
par: Keshav, Sanath, et autres
Publié: (2025)

TMS: Trajectory-Mixed Supervision for Reward-Free, On-Policy SFT
par: Khan, Rana Muhammad Shahroz, et autres
Publié: (2026)

On the Exponential Convergence for Offline RLHF with Pairwise Comparisons
par: Chen, Zhirui, et autres
Publié: (2024)

Data-driven Error Estimation: Excess Risk Bounds without Class Complexity as Input
par: Krishnamurthy, Sanath Kumar, et autres
Publié: (2024)

Information-Theoretic Reward Modeling for Stable RLHF: Detecting and Mitigating Reward Hacking
par: Miao, Yuchun, et autres
Publié: (2025)

WPO: Enhancing RLHF with Weighted Preference Optimization
par: Zhou, Wenxuan, et autres
Publié: (2024)