Imagen de Portada

Online Bandit Learning with Offline Preference Data for Improved RLHF

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Agnihotri, Akhil, Jain, Rahul, Ramachandran, Deepak, Wen, Zheng
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Multi-Objective Preference Optimization: Improving Human Alignment of Generative Models
por: Agnihotri, Akhil, et al.
Publicado: (2025)

Best Policy Learning from Trajectory Preference Feedback
por: Agnihotri, Akhil, et al.
Publicado: (2025)

e-COP : Episodic Constrained Optimization of Policies
por: Agnihotri, Akhil, et al.
Publicado: (2024)

Multi-Objective Reward and Preference Optimization: Theory and Algorithms
por: Agnihotri, Akhil
Publicado: (2025)

ACPO: A Policy Optimization Algorithm for Average MDPs with Constraints
por: Agnihotri, Akhil, et al.
Publicado: (2023)

Offline Constrained RLHF with Multiple Preference Oracles
por: Latham, Brenden, et al.
Publicado: (2026)

Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF
por: Cen, Shicong, et al.
Publicado: (2024)

Efficient Online Learning with Offline Datasets for Infinite Horizon MDPs: A Bayesian Approach
por: Tang, Dengwang, et al.
Publicado: (2023)

Robust LLM Alignment via Distributionally Robust Direct Preference Optimization
por: Xu, Zaiyan, et al.
Publicado: (2025)

Online Bandits with (Biased) Offline Data: Adaptive Learning under Distribution Mismatch
por: Cheung, Wang Chi, et al.
Publicado: (2024)

Offline and Online KL-Regularized RLHF under Differential Privacy
por: Wu, Yulian, et al.
Publicado: (2025)

Online Policy Learning from Offline Preferences
por: Zhang, Guoxi, et al.
Publicado: (2024)

Direction-Aware Offline-to-Online Learning in Linear Contextual Bandits
por: Han, Zean, et al.
Publicado: (2026)

Offline Local Search for Online Stochastic Bandits
por: Benadè, Gerdus, et al.
Publicado: (2026)

Markov Balance Satisfaction Improves Performance in Strictly Batch Offline Imitation Learning
por: Agrawal, Rishabh, et al.
Publicado: (2024)

Regularized Online RLHF with Generalized Bilinear Preferences
por: Lee, Junghyun, et al.
Publicado: (2026)

Policy-labeled Preference Learning: Is Preference Enough for RLHF?
por: Cho, Taehyun, et al.
Publicado: (2025)

DOPL: Direct Online Preference Learning for Restless Bandits with Preference Feedback
por: Xiong, Guojun, et al.
Publicado: (2024)

Provably Mitigating Corruption, Overoptimization, and Verbosity Simultaneously in Offline and Online RLHF/DPO Alignment
por: Chen, Ziyi, et al.
Publicado: (2025)

Sharp Analysis for KL-Regularized Contextual Bandits and RLHF
por: Zhao, Heyang, et al.
Publicado: (2024)

SharedRep-RLHF: A Shared Representation Approach to RLHF with Diverse Preferences
por: Mukherjee, Arpan, et al.
Publicado: (2025)

Offline Learning for Combinatorial Multi-armed Bandits
por: Liu, Xutong, et al.
Publicado: (2025)

Enhancing Online Reinforcement Learning with Meta-Learned Objective from Offline Data
por: Deng, Shilong, et al.
Publicado: (2025)

Balance Equation-based Distributionally Robust Offline Imitation Learning
por: Agrawal, Rishabh, et al.
Publicado: (2025)

Fairness of Exposure in Online Restless Multi-armed Bandits
por: Sood, Archit, et al.
Publicado: (2024)

ENOTO: Improving Offline-to-Online Reinforcement Learning with Q-Ensembles
por: Zhao, Kai, et al.
Publicado: (2023)

KL-regularization Itself is Differentially Private in Bandits and RLHF
por: Zhang, Yizhou, et al.
Publicado: (2025)

When Dynamics Shift, Robust Task Inference Wins: Offline Imitation Learning with Behavior Foundation Models Revisited
por: Agrawal, Rishabh, et al.
Publicado: (2026)

Sharpe Ratio-Guided Active Learning for Preference Optimization in RLHF
por: Belakaria, Syrine, et al.
Publicado: (2025)

Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF
por: Siththaranjan, Anand, et al.
Publicado: (2023)

Leveraging Offline Data in Linear Latent Contextual Bandits
por: Kausik, Chinmaya, et al.
Publicado: (2024)

Online Learning with Improving Agents: Multiclass, Budgeted Agents and Bandit Learners
por: Ashkezari, Sajad, et al.
Publicado: (2026)

RLHF Workflow: From Reward Modeling to Online RLHF
por: Dong, Hanze, et al.
Publicado: (2024)

Bayesian Regret Minimization in Offline Bandits
por: Petrik, Marek, et al.
Publicado: (2023)

Group-Sensitive Offline Contextual Bandits
por: Guo, Yihong, et al.
Publicado: (2025)

Offline Clustering of Linear Bandits: The Power of Clusters under Limited Data
por: Liu, Jingyuan, et al.
Publicado: (2025)

Active Advantage-Aligned Online Reinforcement Learning with Offline Data
por: Liu, Xuefeng, et al.
Publicado: (2025)

Adaptive Margin RLHF via Preference over Preferences
por: Chittepu, Yaswanth, et al.
Publicado: (2025)

Improved Offline Contextual Bandits with Second-Order Bounds: Betting and Freezing
por: Ryu, J. Jon, et al.
Publicado: (2025)

On the Exponential Convergence for Offline RLHF with Pairwise Comparisons
por: Chen, Zhirui, et al.
Publicado: (2024)