Cover Image

The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human Feedback

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lambert, Nathan, Calandra, Roberto
Format:	Preprint
Published:	2023
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Unified View on Solving Objective Mismatch in Model-Based Reinforcement Learning
by: Wei, Ran, et al.
Published: (2023)

Reinforcement Learning from Human Feedback
by: Lambert, Nathan
Published: (2025)

PARL: A Unified Framework for Policy Alignment in Reinforcement Learning from Human Feedback
by: Chakraborty, Souradip, et al.
Published: (2023)

Strategyproof Reinforcement Learning from Human Feedback
by: Buening, Thomas Kleine, et al.
Published: (2025)

Apple: Toward General Active Perception via Reinforcement Learning
by: Schneider, Tim, et al.
Published: (2025)

Learning Gentle Grasping Using Vision, Sound, and Touch
by: Nakahara, Ken, et al.
Published: (2025)

SAFE: Stable Alignment Finetuning with Entropy-Aware Predictive Control for Reinforcement Learning from Human Feedback (RLHF)
by: Maity, Dipan
Published: (2026)

Robust Reinforcement Learning from Corrupted Human Feedback
by: Bukharin, Alexander, et al.
Published: (2024)

Dual Active Learning for Reinforcement Learning from Human Feedback
by: Liu, Pangpang, et al.
Published: (2024)

Constrained Reinforcement Learning Under Model Mismatch
by: Sun, Zhongchang, et al.
Published: (2024)

Understanding the Learning Dynamics of Alignment with Human Feedback
by: Im, Shawn, et al.
Published: (2024)

Breaking the Performance Ceiling in Reinforcement Learning requires Inference Strategies
by: Chalumeau, Felix, et al.
Published: (2025)

FBOS-RL: Feedback-Driven Bi-Objective Synergistic Reinforcement Learning
by: Zhang, Xikai, et al.
Published: (2026)

Reinforcement Learning from Multi-level and Episodic Human Feedback
by: Elahi, Muhammad Qasim, et al.
Published: (2025)

Reinforcement Learning from Human Feedback: A Statistical Perspective
by: Liu, Pangpang, et al.
Published: (2026)

A Minimaximalist Approach to Reinforcement Learning from Human Feedback
by: Swamy, Gokul, et al.
Published: (2024)

Dense Reward for Free in Reinforcement Learning from Human Feedback
by: Chan, Alex J., et al.
Published: (2024)

Multi-turn Reinforcement Learning from Preference Human Feedback
by: Shani, Lior, et al.
Published: (2024)

Alignment Tampering: How Reinforcement Learning from Human Feedback Is Exploited to Optimize Misaligned Biases
by: Hahm, Dongyoon, et al.
Published: (2026)

The Power of Active Multi-Task Learning in Reinforcement Learning from Human Feedback
by: Chen, Ruitao, et al.
Published: (2024)

Low-Rank Contextual Reinforcement Learning from Heterogeneous Human Feedback
by: Lee, Seong Jin, et al.
Published: (2024)

Data-dependent Exploration for Online Reinforcement Learning from Human Feedback
by: Zhang, Zhen-Yu, et al.
Published: (2026)

Provable Reinforcement Learning from Human Feedback with an Unknown Link Function
by: Zhang, Qining, et al.
Published: (2025)

A Survey of Reinforcement Learning from Human Feedback
by: Kaufmann, Timo, et al.
Published: (2023)

Online Iterative Reinforcement Learning from Human Feedback with General Preference Model
by: Ye, Chenlu, et al.
Published: (2024)

TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback
by: Pang, Lei, et al.
Published: (2025)

Exploring Data Scaling Trends and Effects in Reinforcement Learning from Human Feedback
by: Shen, Wei, et al.
Published: (2025)

Parameter Efficient Reinforcement Learning from Human Feedback
by: Sidahmed, Hakim, et al.
Published: (2024)

Adaptive Preference Scaling for Reinforcement Learning with Human Feedback
by: Hong, Ilgee, et al.
Published: (2024)

Corruption Robust Offline Reinforcement Learning with Human Feedback
by: Mandal, Debmalya, et al.
Published: (2024)

RLTHF: Targeted Human Feedback for LLM Alignment
by: Xu, Yifei, et al.
Published: (2025)

Understanding Uncertainty-based Active Learning Under Model Mismatch
by: Rahmati, Amir Hossein, et al.
Published: (2024)

Multi-Objective Preference Optimization: Improving Human Alignment of Generative Models
by: Agnihotri, Akhil, et al.
Published: (2025)

Swap-guided Preference Learning for Personalized Reinforcement Learning from Human Feedback
by: Kim, Gihoon, et al.
Published: (2026)

Distributionally Robust Reinforcement Learning with Human Feedback
by: Mandal, Debmalya, et al.
Published: (2025)

Privacy-Preserving Reinforcement Learning from Human Feedback via Decoupled Reward Modeling
by: Cho, Young Hyun, et al.
Published: (2026)

Policy Gradient Primal-Dual Method for Safe Reinforcement Learning from Human Feedback
by: Liu, Qiang, et al.
Published: (2026)

Sample-Efficient Reinforcement Learning from Human Feedback via Information-Directed Sampling
by: Qi, Han, et al.
Published: (2025)

RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback
by: Lee, Harrison, et al.
Published: (2023)

Towards User-level Private Reinforcement Learning with Human Feedback
by: Zhang, Jiaming, et al.
Published: (2025)