Image de couverture de livre

Aligning LLMs Toward Multi-Turn Conversational Outcomes Using Iterative PPO

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Jiang, Daniel R., Bhandari, Jalaj, Yang, Yukai, Munos, Rémi, Lu, Tyler
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Turn-PPO: Turn-Level Advantage Estimation with PPO for Improved Multi-Turn RL in Agentic LLMs
par: Li, Junbo, et autres
Publié: (2025)

Aligned Multi Objective Optimization
par: Efroni, Yonathan, et autres
Publié: (2025)

Outcome-based Exploration for LLM Reasoning
par: Song, Yuda, et autres
Publié: (2025)

On a few pitfalls in KL divergence gradient estimation for RL
par: Tang, Yunhao, et autres
Publié: (2025)

Super-Exponential Regret for UCT, AlphaGo and Variants
par: Orseau, Laurent, et autres
Publié: (2024)

Efficient RL Training for LLMs with Experience Replay
par: Arnal, Charles, et autres
Publié: (2026)

Bandits attack function optimization
par: Preux, Philippe, et autres
Publié: (2026)

Stochastic simultaneous optimistic optimization
par: Valko, Michal, et autres
Publié: (2026)

RL-finetuning LLMs from on- and off-policy data with a single algorithm
par: Tang, Yunhao, et autres
Publié: (2025)

Black-box optimization of noisy functions with unknown smoothness
par: Grill, Jean-Bastien, et autres
Publié: (2026)

Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning
par: Grill, Jean-Bastien, et autres
Publié: (2026)

Beyond Verifiable Rewards: Scaling Reinforcement Learning for Language Models to Unverifiable Data
par: Tang, Yunhao, et autres
Publié: (2025)

Optimizing Language Models for Inference Time Objectives using Reinforcement Learning
par: Tang, Yunhao, et autres
Publié: (2025)

Spectral Thompson sampling
par: Kocak, Tomas, et autres
Publié: (2026)

VA-learning as a more efficient alternative to Q-learning
par: Tang, Yunhao, et autres
Publié: (2023)

Spectral bandits for smooth graph functions
par: Valko, Michal, et autres
Publié: (2026)

Efficient learning by implicit exploration in bandit problems with side observations
par: Kocak, Tomas, et autres
Publié: (2026)

Enhancing PPO with Trajectory-Aware Hybrid Policies
par: Liu, Qisai, et autres
Publié: (2025)

Mitigating Conversational Inertia in Multi-Turn Agents
par: Wan, Yang, et autres
Publié: (2026)

Building Math Agents with Multi-Turn Iterative Preference Learning
par: Xiong, Wei, et autres
Publié: (2024)

Turn-Based Structural Triggers: Prompt-Free Backdoors in Multi-Turn LLMs
par: Lu, Yiyang, et autres
Publié: (2026)

Eliciting Behaviors in Multi-Turn Conversations
par: Huang, Jing, et autres
Publié: (2025)

Self-Rewarding PPO: Aligning Large Language Models with Demonstrations Only
par: Zhang, Qingru, et autres
Publié: (2025)

Spectral bandits for smooth graph functions with applications in recommender systems
par: Kocák, Tomáš, et autres
Publié: (2026)

Off-policy Distributional Q($λ$): Distributional RL without Importance Sampling
par: Tang, Yunhao, et autres
Publié: (2024)

Multi-Turn Reasoning LLMs for Task Offloading in Mobile Edge Computing
par: Yang, Ning, et autres
Publié: (2026)

Aligning Frozen LLMs by Reinforcement Learning: An Iterative Reweight-then-Optimize Approach
par: Zhang, Xinnan, et autres
Publié: (2025)

Sampling Complexity of TD and PPO in RKHS
par: Zou, Lu, et autres
Publié: (2025)

Fix Initial Codes and Iteratively Refine Textual Directions Toward Safe Multi-Turn Code Correction
par: Tanaka, Yuto, et autres
Publié: (2026)

Asking Forever: Universal Activations Behind Turn Amplification in Conversational LLMs
par: Coalson, Zachary, et autres
Publié: (2026)

Spectral bandits
par: Kocák, Tomáš, et autres
Publié: (2026)

VinePPO: Refining Credit Assignment in RL Training of LLMs
par: Kazemnejad, Amirhossein, et autres
Publié: (2024)

Planning in entropy-regularized Markov decision processes and games
par: Grill, Jean-Bastien, et autres
Publié: (2026)

Near-Minimax-Optimal Distributional Reinforcement Learning with a Generative Model
par: Rowland, Mark, et autres
Publié: (2024)

Directional-Clamp PPO
par: Karpel, Gilad, et autres
Publié: (2025)

How Many Iterations to Jailbreak? Dynamic Budget Allocation for Multi-Turn LLM Evaluation
par: Feldman, Shai, et autres
Publié: (2026)

Temporal Graph Network: Hallucination Detection in Multi-Turn Conversation
par: Rathore, Vidhi, et autres
Publié: (2026)

How Do LLMs Persuade? Linear Probes Can Uncover Persuasion Dynamics in Multi-Turn Conversations
par: Jaipersaud, Brandon, et autres
Publié: (2025)

Visual Memory Injection Attacks for Multi-Turn Conversations
par: Schlarmann, Christian, et autres
Publié: (2026)

Asymmetric REINFORCE for off-Policy Reinforcement Learning: Balancing positive and negative rewards
par: Arnal, Charles, et autres
Publié: (2025)