Cover Image

Optimizing Language Models for Inference Time Objectives using Reinforcement Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tang, Yunhao, Zheng, Kunhao, Synnaeve, Gabriel, Munos, Rémi
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Soft Policy Optimization: Online Off-Policy RL for Sequence Models
by: Cohen, Taco, et al.
Published: (2025)

Beyond Verifiable Rewards: Scaling Reinforcement Learning for Language Models to Unverifiable Data
by: Tang, Yunhao, et al.
Published: (2025)

On a few pitfalls in KL divergence gradient estimation for RL
by: Tang, Yunhao, et al.
Published: (2025)

Near-Minimax-Optimal Distributional Reinforcement Learning with a Generative Model
by: Rowland, Mark, et al.
Published: (2024)

VA-learning as a more efficient alternative to Q-learning
by: Tang, Yunhao, et al.
Published: (2023)

Asymmetric REINFORCE for off-Policy Reinforcement Learning: Balancing positive and negative rewards
by: Arnal, Charles, et al.
Published: (2025)

Off-policy Distributional Q($λ$): Distributional RL without Importance Sampling
by: Tang, Yunhao, et al.
Published: (2024)

RL-finetuning LLMs from on- and off-policy data with a single algorithm
by: Tang, Yunhao, et al.
Published: (2025)

Super-Exponential Regret for UCT, AlphaGo and Variants
by: Orseau, Laurent, et al.
Published: (2024)

Extrapolative Weight Averaging Reveals Correctness-Efficiency Frontiers in Code RL
by: Zheng, Kunhao, et al.
Published: (2026)

Offline Regularised Reinforcement Learning for Large Language Models Alignment
by: Richemond, Pierre Harvey, et al.
Published: (2024)

An Analysis of Quantile Temporal-Difference Learning
by: Rowland, Mark, et al.
Published: (2023)

Bandits attack function optimization
by: Preux, Philippe, et al.
Published: (2026)

Stochastic simultaneous optimistic optimization
by: Valko, Michal, et al.
Published: (2026)

Generalized Preference Optimization: A Unified Approach to Offline Alignment
by: Tang, Yunhao, et al.
Published: (2024)

Black-box optimization of noisy functions with unknown smoothness
by: Grill, Jean-Bastien, et al.
Published: (2026)

Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning
by: Grill, Jean-Bastien, et al.
Published: (2026)

Outcome-based Exploration for LLM Reasoning
by: Song, Yuda, et al.
Published: (2025)

Set Block Decoding is a Language Model Inference Accelerator
by: Gat, Itai, et al.
Published: (2025)

Expanding the Capabilities of Reinforcement Learning via Text Feedback
by: Song, Yuda, et al.
Published: (2026)

Spectral Thompson sampling
by: Kocak, Tomas, et al.
Published: (2026)

Spectral bandits for smooth graph functions
by: Valko, Michal, et al.
Published: (2026)

Efficient learning by implicit exploration in bandit problems with side observations
by: Kocak, Tomas, et al.
Published: (2026)

Human Alignment of Large Language Models through Online Preference Optimisation
by: Calandriello, Daniele, et al.
Published: (2024)

Spectral bandits for smooth graph functions with applications in recommender systems
by: Kocák, Tomáš, et al.
Published: (2026)

Efficient RL Training for LLMs with Experience Replay
by: Arnal, Charles, et al.
Published: (2026)

Understanding the performance gap between online and offline alignment algorithms
by: Tang, Yunhao, et al.
Published: (2024)

Multi-Objective Reinforcement Learning for Large Language Model Optimization: Visionary Perspective
by: Kong, Lingxiao, et al.
Published: (2025)

Model-free Posterior Sampling via Learning Rate Randomization
by: Tiapkin, Daniil, et al.
Published: (2023)

Aligning LLMs Toward Multi-Turn Conversational Outcomes Using Iterative PPO
by: Jiang, Daniel R., et al.
Published: (2025)

Beyond Inference-Time Search: Reinforcement Learning Synthesizes Reusable Solvers
by: Massoudi, Soheyl, et al.
Published: (2026)

DROGO: Default Representation Objective via Graph Optimization in Reinforcement Learning
by: Tse, Hon Tik, et al.
Published: (2026)

Multi-turn Reinforcement Learning from Preference Human Feedback
by: Shani, Lior, et al.
Published: (2024)

Spectral bandits
by: Kocák, Tomáš, et al.
Published: (2026)

Multi-Objective Hierarchical Optimization with Large Language Models
by: Schwanke, Andrej, et al.
Published: (2026)

Inference-Time Alignment of Diffusion Models with Direct Noise Optimization
by: Tang, Zhiwei, et al.
Published: (2024)

Decoding-Time Language Model Alignment with Multiple Objectives
by: Shi, Ruizhe, et al.
Published: (2024)

Planning in entropy-regularized Markov decision processes and games
by: Grill, Jean-Bastien, et al.
Published: (2026)

Enhancing Online Reinforcement Learning with Meta-Learned Objective from Offline Data
by: Deng, Shilong, et al.
Published: (2025)

T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
by: Hou, Zhenyu, et al.
Published: (2025)