Cover Image

Stabilizing Policy Gradients for Sample-Efficient Reinforcement Learning in LLM Reasoning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Melo, Luckeciano C., Abate, Alessandro, Gal, Yarin
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Temporal-Difference Variational Continual Learning
by: Melo, Luckeciano C., et al.
Published: (2024)

Deep Bayesian Active Learning for Preference Modeling in Large Language Models
by: Melo, Luckeciano C., et al.
Published: (2024)

Iterative Deployment Improves Planning Skills in LLMs
by: Corrêa, Augusto B., et al.
Published: (2025)

Partial Identifiability and Misspecification in Inverse Reinforcement Learning
by: Skalse, Joar, et al.
Published: (2024)

Partial Identifiability in Inverse Reinforcement Learning For Agents With Non-Exponential Discounting
by: Skalse, Joar, et al.
Published: (2024)

Gradient Boosting Reinforcement Learning
by: Fuhrer, Benjamin, et al.
Published: (2024)

DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RL
by: Jackermeier, Mathias, et al.
Published: (2024)

In-Context Learning Learns Label Relationships but Is Not Conventional Learning
by: Kossen, Jannik, et al.
Published: (2023)

Policy Gradient with Tree Expansion
by: Dalal, Gal, et al.
Published: (2023)

Efficient Solution and Learning of Robust Factored MDPs
by: Schnitzer, Yannik, et al.
Published: (2025)

Simple Baselines are Competitive with Code Evolution
by: Gideoni, Yonatan, et al.
Published: (2026)

Probabilistic Performance Guarantees for Multi-Task Reinforcement Learning
by: Schnitzer, Yannik, et al.
Published: (2026)

When is Offline Policy Selection Sample Efficient for Reinforcement Learning?
by: Liu, Vincent, et al.
Published: (2023)

Do Multilingual LLMs Think In English?
by: Schut, Lisa, et al.
Published: (2025)

Can LLMs Guide Their Own Exploration? Gradient-Guided Reinforcement Learning for LLM Reasoning
by: Liang, Zhenwen, et al.
Published: (2025)

MASPO: Unifying Gradient Utilization, Probability Mass, and Signal Reliability for Robust and Sample-Efficient LLM Reasoning
by: Fu, Xiaoliang, et al.
Published: (2026)

Proximal Policy Gradient Arborescence for Quality Diversity Reinforcement Learning
by: Batra, Sumeet, et al.
Published: (2023)

Certifiably Robust Policies for Uncertain Parametric Environments
by: Schnitzer, Yannik, et al.
Published: (2024)

Latent-Space Contrastive Reinforcement Learning for Stable and Efficient LLM Reasoning
by: Shan, Lianlei, et al.
Published: (2026)

Return Capping: Sample-Efficient CVaR Policy Gradient Optimisation
by: Mead, Harry, et al.
Published: (2025)

Policy Gradient Methods for Non-Markovian Reinforcement Learning
by: Kar, Avik, et al.
Published: (2026)

Policy Gradients for Cumulative Prospect Theory in Reinforcement Learning
by: Lepel, Olivier, et al.
Published: (2024)

On the Limitations of Markovian Rewards to Express Multi-Objective, Risk-Sensitive, and Modal Tasks
by: Skalse, Joar, et al.
Published: (2024)

Semantically Labelled Automata for Multi-Task Reinforcement Learning with LTL Instructions
by: Abate, Alessandro, et al.
Published: (2026)

SPEQ: Offline Stabilization Phases for Efficient Q-Learning in High Update-To-Data Ratio Reinforcement Learning
by: Romeo, Carlo, et al.
Published: (2025)

Rethinking the Sampling Criteria in Reinforcement Learning for LLM Reasoning: A Competence-Difficulty Alignment Perspective
by: Kong, Deyang, et al.
Published: (2025)

$K$-Level Policy Gradients for Multi-Agent Reinforcement Learning
by: Reddi, Aryaman, et al.
Published: (2025)

Emergence of Exploration in Policy Gradient Reinforcement Learning via Retrying
by: Nishimori, Soichiro, et al.
Published: (2026)

Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities
by: Nikitin, Alexander, et al.
Published: (2024)

Stable Reinforcement Learning for Efficient Reasoning
by: Dai, Muzhi, et al.
Published: (2025)

On the Design of KL-Regularized Policy Gradient Algorithms for LLM Reasoning
by: Zhang, Yifan, et al.
Published: (2025)

Interpreting and Controlling LLM Reasoning through Integrated Policy Gradient
by: Li, Changming, et al.
Published: (2026)

On the Global Optimality of Policy Gradient Methods in General Utility Reinforcement Learning
by: Barakat, Anas, et al.
Published: (2024)

PG-Rainbow: Using Distributional Reinforcement Learning in Policy Gradient Methods
by: Jeon, WooJae, et al.
Published: (2024)

Reasoning Introduces New Poisoning Attacks Yet Makes Them More Complicated
by: Foerster, Hanna, et al.
Published: (2025)

Rethinking Policy Diversity in Ensemble Policy Gradient in Large-Scale Reinforcement Learning
by: Shitanda, Naoki, et al.
Published: (2026)

Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts
by: Zheng, Haizhong, et al.
Published: (2025)

Unearthing Gems from Stones: Policy Optimization with Negative Sample Augmentation for LLM Reasoning
by: Yang, Zhaohui, et al.
Published: (2025)

RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning
by: Mao, Yixiu, et al.
Published: (2026)

LearnAlign: Data Selection for LLM Reinforcement Learning with Improved Gradient Alignment
by: Li, Shipeng, et al.
Published: (2025)