Cover Image

Human Alignment of Large Language Models through Online Preference Optimisation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Calandriello, Daniele, Guo, Daniel, Munos, Remi, Rowland, Mark, Tang, Yunhao, Pires, Bernardo Avila, Richemond, Pierre Harvey, Lan, Charline Le, Valko, Michal, Liu, Tianqi, Joshi, Rishabh, Zheng, Zeyu, Piot, Bilal
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Generalized Preference Optimization: A Unified Approach to Offline Alignment
by: Tang, Yunhao, et al.
Published: (2024)

Offline Regularised Reinforcement Learning for Large Language Models Alignment
by: Richemond, Pierre Harvey, et al.
Published: (2024)

VA-learning as a more efficient alternative to Q-learning
by: Tang, Yunhao, et al.
Published: (2023)

Off-policy Distributional Q($λ$): Distributional RL without Importance Sampling
by: Tang, Yunhao, et al.
Published: (2024)

Understanding the performance gap between online and offline alignment algorithms
by: Tang, Yunhao, et al.
Published: (2024)

Large-scale semi-supervised learning with online spectral graph sparsification
by: Calandriello, Daniele, et al.
Published: (2026)

Analysis of Nystrom method with sequential ridge leverage scores
by: Calandriello, Daniele, et al.
Published: (2026)

Pack only the essentials: Adaptive dictionary learning for kernel ridge regression
by: Calandriello, Daniele, et al.
Published: (2026)

Bandits attack function optimization
by: Preux, Philippe, et al.
Published: (2026)

Stochastic simultaneous optimistic optimization
by: Valko, Michal, et al.
Published: (2026)

RL-finetuning LLMs from on- and off-policy data with a single algorithm
by: Tang, Yunhao, et al.
Published: (2025)

Black-box optimization of noisy functions with unknown smoothness
by: Grill, Jean-Bastien, et al.
Published: (2026)

Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning
by: Grill, Jean-Bastien, et al.
Published: (2026)

Building Math Agents with Multi-Turn Iterative Preference Learning
by: Xiong, Wei, et al.
Published: (2024)

Improved large-scale graph learning through ridge spectral sparsification
by: Calandriello, Daniele, et al.
Published: (2026)

Nash Learning from Human Feedback
by: Munos, Rémi, et al.
Published: (2023)

Model-free Posterior Sampling via Learning Rate Randomization
by: Tiapkin, Daniil, et al.
Published: (2023)

Spectral Thompson sampling
by: Kocak, Tomas, et al.
Published: (2026)

Spectral bandits for smooth graph functions
by: Valko, Michal, et al.
Published: (2026)

Efficient learning by implicit exploration in bandit problems with side observations
by: Kocak, Tomas, et al.
Published: (2026)

Multi-turn Reinforcement Learning from Preference Human Feedback
by: Shani, Lior, et al.
Published: (2024)

On a few pitfalls in KL divergence gradient estimation for RL
by: Tang, Yunhao, et al.
Published: (2025)

Spectral bandits for smooth graph functions with applications in recommender systems
by: Kocák, Tomáš, et al.
Published: (2026)

Spectral bandits
by: Kocák, Tomáš, et al.
Published: (2026)

Near-Minimax-Optimal Distributional Reinforcement Learning with a Generative Model
by: Rowland, Mark, et al.
Published: (2024)

Planning in entropy-regularized Markov decision processes and games
by: Grill, Jean-Bastien, et al.
Published: (2026)

Optimizing Return Distributions with Distributional Dynamic Programming
by: Pires, Bernardo Ávila, et al.
Published: (2025)

Beyond Verifiable Rewards: Scaling Reinforcement Learning for Language Models to Unverifiable Data
by: Tang, Yunhao, et al.
Published: (2025)

Optimizing Language Models for Inference Time Objectives using Reinforcement Learning
by: Tang, Yunhao, et al.
Published: (2025)

Soft Policy Optimization: Online Off-Policy RL for Sequence Models
by: Cohen, Taco, et al.
Published: (2025)

On two ways to use determinantal point processes for Monte Carlo integration
by: Gautier, Guillaume, et al.
Published: (2026)

Online combinatorial optimization with stochastic decision sets and adversarial losses
by: Neu, Gergely, et al.
Published: (2026)

Proximal Point Nash Learning from Human Feedback
by: Tiapkin, Daniil, et al.
Published: (2025)

Demonstration-Regularized RL
by: Tiapkin, Daniil, et al.
Published: (2023)

Direct Language Model Alignment from Online AI Feedback
by: Guo, Shangmin, et al.
Published: (2024)

Super-Exponential Regret for UCT, AlphaGo and Variants
by: Orseau, Laurent, et al.
Published: (2024)

Bandits on graphs and structures
by: Valko, Michal
Published: (2026)

Adaptive graph-based algorithms for conditional anomaly detection and semi-supervised learning
by: Valko, Michal
Published: (2026)

Online learning with noisy side observations
by: Kocák, Tomáš, et al.
Published: (2026)

Online learning with Erdős-Rényi side-observation graphs
by: Kocák, Tomáš, et al.
Published: (2026)