Imagen de Portada

Offline Regularised Reinforcement Learning for Large Language Models Alignment

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Richemond, Pierre Harvey, Tang, Yunhao, Guo, Daniel, Calandriello, Daniele, Azar, Mohammad Gheshlaghi, Rafailov, Rafael, Pires, Bernardo Avila, Tarassov, Eugene, Spangher, Lucas, Ellsworth, Will, Severyn, Aliaksei, Mallinson, Jonathan, Shani, Lior, Shamir, Gil, Joshi, Rishabh, Liu, Tianqi, Munos, Remi, Piot, Bilal
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Generalized Preference Optimization: A Unified Approach to Offline Alignment
por: Tang, Yunhao, et al.
Publicado: (2024)

Human Alignment of Large Language Models through Online Preference Optimisation
por: Calandriello, Daniele, et al.
Publicado: (2024)

Understanding the performance gap between online and offline alignment algorithms
por: Tang, Yunhao, et al.
Publicado: (2024)

West-of-N: Synthetic Preferences for Self-Improving Reward Models
por: Pace, Alizée, et al.
Publicado: (2024)

Multi-turn Reinforcement Learning from Preference Human Feedback
por: Shani, Lior, et al.
Publicado: (2024)

On a few pitfalls in KL divergence gradient estimation for RL
por: Tang, Yunhao, et al.
Publicado: (2025)

Nash Learning from Human Feedback
por: Munos, Rémi, et al.
Publicado: (2023)

Small Language Models Improve Giants by Rewriting Their Outputs
por: Vernikos, Giorgos, et al.
Publicado: (2023)

An Analysis of Quantile Temporal-Difference Learning
por: Rowland, Mark, et al.
Publicado: (2023)

Off-policy Distributional Q($λ$): Distributional RL without Importance Sampling
por: Tang, Yunhao, et al.
Publicado: (2024)

Reproducible empirical evidence of cosmological-scale asymmetry in galaxy spin directions: comment on arXiv:2404.06617
por: Shamir, Lior
Publicado: (2024)

Asymmetry in the distribution of HSC galaxy spin directions: comment on arXiv:2410.18884v1
por: Shamir, Lior
Publicado: (2024)

Asymmetry in galaxy spin directions: a fully reproducible experiment using HSC data
por: Shamir, Lior
Publicado: (2024)

The distribution of galaxy rotation in JWST Advanced Deep Extragalactic Survey
por: Shamir, Lior
Publicado: (2025)

A simple direct empirical observation of systematic bias of the redshift as a distance indicator
por: Shamir, Lior
Publicado: (2023)

An empirical consistent redshift bias: A possible direct reproducible observation of Zwicky's TL theory
por: Shamir, Lior
Publicado: (2024)

Galaxy spin direction asymmetry in JWST deep fields
por: Shamir, Lior
Publicado: (2024)

Beyond Verifiable Rewards: Scaling Reinforcement Learning for Language Models to Unverifiable Data
por: Tang, Yunhao, et al.
Publicado: (2025)

VA-learning as a more efficient alternative to Q-learning
por: Tang, Yunhao, et al.
Publicado: (2023)

Optimizing Language Models for Inference Time Objectives using Reinforcement Learning
por: Tang, Yunhao, et al.
Publicado: (2025)

Self-Improving Robust Preference Optimization
por: Choi, Eugene, et al.
Publicado: (2024)

Building Math Agents with Multi-Turn Iterative Preference Learning
por: Xiong, Wei, et al.
Publicado: (2024)

Super-Exponential Regret for UCT, AlphaGo and Variants
por: Orseau, Laurent, et al.
Publicado: (2024)

An open dataset of neural networks for hypernetwork research
por: Kurtenbach, David, et al.
Publicado: (2025)

A data science and machine learning approach to continuous analysis of Shakespeare's plays
por: Swisher, Charles, et al.
Publicado: (2023)

RL-finetuning LLMs from on- and off-policy data with a single algorithm
por: Tang, Yunhao, et al.
Publicado: (2025)

Near-Minimax-Optimal Distributional Reinforcement Learning with a Generative Model
por: Rowland, Mark, et al.
Publicado: (2024)

Asymmetric REINFORCE for off-Policy Reinforcement Learning: Balancing positive and negative rewards
por: Arnal, Charles, et al.
Publicado: (2025)

Galaxy image simplification using Generative AI
por: Erukude, Sai Teja, et al.
Publicado: (2025)

Analysis and prevention of AI-based phishing email attacks
por: Eze, Chibuike Samuel, et al.
Publicado: (2024)

Model-free Posterior Sampling via Learning Rate Randomization
por: Tiapkin, Daniil, et al.
Publicado: (2023)

Soft Policy Optimization: Online Off-Policy RL for Sequence Models
por: Cohen, Taco, et al.
Publicado: (2025)

Bandits attack function optimization
por: Preux, Philippe, et al.
Publicado: (2026)

Stochastic simultaneous optimistic optimization
por: Valko, Michal, et al.
Publicado: (2026)

Outcome-based Exploration for LLM Reasoning
por: Song, Yuda, et al.
Publicado: (2025)

ShiQ: Bringing back Bellman to LLMs
por: Clavier, Pierre, et al.
Publicado: (2025)

Black-box optimization of noisy functions with unknown smoothness
por: Grill, Jean-Bastien, et al.
Publicado: (2026)

Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning
por: Grill, Jean-Bastien, et al.
Publicado: (2026)

CornViT: A Multi-Stage Convolutional Vision Transformer Framework for Hierarchical Corn Kernel Analysis
por: Erukude, Sai Teja, et al.
Publicado: (2025)

Identifying Bias in Deep Neural Networks Using Image Transforms
por: Erukude, Sai Teja, et al.
Publicado: (2024)