Copertina

Hybrid Preference Optimization for Alignment: Provably Faster Convergence Rates by Combining Offline Preferences with Online Exploration

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Bose, Avinandan, Xiong, Zhihan, Saha, Aadirupa, Du, Simon Shaolei, Fazel, Maryam
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Offline Multi-task Transfer RL with Representational Penalization
di: Bose, Avinandan, et al.
Pubblicazione: (2024)

LoRe: Personalizing LLMs via Low-Rank Reward Modeling
di: Bose, Avinandan, et al.
Pubblicazione: (2025)

Extragradient Preference Optimization (EGPO): Beyond Last-Iterate Convergence for Nash Learning from Human Feedback
di: Zhou, Runlong, et al.
Pubblicazione: (2025)

Unregularized Linear Convergence in Zero-Sum Game from Preference Feedback
di: Chen, Shulun, et al.
Pubblicazione: (2025)

DP-Dueling: Learning from Preference Feedback without Compromising User Privacy
di: Saha, Aadirupa, et al.
Pubblicazione: (2024)

Keeping up with dynamic attackers: Certifying robustness to adaptive online data poisoning
di: Bose, Avinandan, et al.
Pubblicazione: (2025)

Offline congestion games: How feedback type affects data coverage requirement
di: Jiang, Haozhe, et al.
Pubblicazione: (2022)

Dual Approximation Policy Optimization
di: Xiong, Zhihan, et al.
Pubblicazione: (2024)

Convergence Dynamics of Over-Parameterized Score Matching for a Single Gaussian
di: Zhang, Yiran, et al.
Pubblicazione: (2025)

Global Convergence of Four-Layer Matrix Factorization under Random Initialization
di: Luo, Minrui, et al.
Pubblicazione: (2025)

Cold-Start Personalization via Training-Free Priors from Structured World Models
di: Bose, Avinandan, et al.
Pubblicazione: (2026)

Toward Global Convergence of Gradient EM for Over-Parameterized Gaussian Mixture Models
di: Xu, Weihang, et al.
Pubblicazione: (2024)

Stop Relying on No-Choice and Do not Repeat the Moves: Optimal, Efficient and Practical Algorithms for Assortment Optimization
di: Saha, Aadirupa, et al.
Pubblicazione: (2024)

Understanding the Performance Gap in Preference Learning: A Dichotomy of RLHF and DPO
di: Shi, Ruizhe, et al.
Pubblicazione: (2025)

Global Convergence of Gradient EM for Over-Parameterized Gaussian Mixtures
di: Zhou, Mo, et al.
Pubblicazione: (2025)

Direct Preference Optimization with Rating Information: Practical Algorithms and Provable Gains
di: Viano, Luca, et al.
Pubblicazione: (2026)

The Crucial Role of Samplers in Online Direct Preference Optimization
di: Shi, Ruizhe, et al.
Pubblicazione: (2024)

A Black-box Approach for Non-stationary Multi-agent Reinforcement Learning
di: Jiang, Haozhe, et al.
Pubblicazione: (2023)

Generalized Preference Optimization: A Unified Approach to Offline Alignment
di: Tang, Yunhao, et al.
Pubblicazione: (2024)

Online Preference Alignment for Language Models via Count-based Exploration
di: Bai, Chenjia, et al.
Pubblicazione: (2025)

Online Policy Learning from Offline Preferences
di: Zhang, Guoxi, et al.
Pubblicazione: (2024)

On The Complexity of Best-Arm Identification in Non-Stationary Linear Bandits
di: Maynard-Zhang, Leo, et al.
Pubblicazione: (2026)

Self-Consistency Preference Optimization
di: Prasad, Archiki, et al.
Pubblicazione: (2024)

Self-Exploring Language Models: Active Preference Elicitation for Online Alignment
di: Zhang, Shenao, et al.
Pubblicazione: (2024)

A/B Testing and Best-arm Identification for Linear Bandits with Robustness to Non-stationarity
di: Xiong, Zhihan, et al.
Pubblicazione: (2023)

Policy-Based Trajectory Clustering in Offline Reinforcement Learning
di: Hu, Hao, et al.
Pubblicazione: (2025)

Initializing Services in Interactive ML Systems for Diverse Users
di: Bose, Avinandan, et al.
Pubblicazione: (2023)

Online Bandit Learning with Offline Preference Data for Improved RLHF
di: Agnihotri, Akhil, et al.
Pubblicazione: (2024)

Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF
di: Cen, Shicong, et al.
Pubblicazione: (2024)

One Good Source is All You Need: Near-Optimal Regret for Bandits under Heterogeneous Noise
di: Bhat, Amith, et al.
Pubblicazione: (2026)

LLM-as-Judge on a Budget
di: Saha, Aadirupa, et al.
Pubblicazione: (2026)

Tracking the Best Expert Privately
di: Saha, Aadirupa, et al.
Pubblicazione: (2025)

Latent Adversarial Regularization for Offline Preference Optimization
di: Jiang, Enyi, et al.
Pubblicazione: (2026)

Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences
di: Ferbach, Damien, et al.
Pubblicazione: (2024)

Transformers are Efficient Compilers, Provably
di: Zhai, Xiyu, et al.
Pubblicazione: (2024)

Energy-Based Preference Model Offers Better Offline Alignment than the Bradley-Terry Preference Model
di: Hong, Yuzhong, et al.
Pubblicazione: (2024)

Provably Convergent Primal-Dual DPO for Constrained LLM Alignment
di: Du, Yihan, et al.
Pubblicazione: (2025)

Square$χ$PO: Differentially Private and Robust $χ^2$-Preference Optimization in Offline Direct Alignment
di: Zhou, Xingyu, et al.
Pubblicazione: (2025)

OPRIDE: Offline Preference-based Reinforcement Learning via In-Dataset Exploration
di: Yang, Yiqin, et al.
Pubblicazione: (2026)

Distortion of AI Alignment: Does Preference Optimization Optimize for Preferences?
di: Gölz, Paul, et al.
Pubblicazione: (2025)