Copertina

Off-policy Distributional Q($λ$): Distributional RL without Importance Sampling

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Tang, Yunhao, Rowland, Mark, Munos, Rémi, Pires, Bernardo Ávila, Dabney, Will
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Near-Minimax-Optimal Distributional Reinforcement Learning with a Generative Model
di: Rowland, Mark, et al.
Pubblicazione: (2024)

VA-learning as a more efficient alternative to Q-learning
di: Tang, Yunhao, et al.
Pubblicazione: (2023)

Optimizing Return Distributions with Distributional Dynamic Programming
di: Pires, Bernardo Ávila, et al.
Pubblicazione: (2025)

On a few pitfalls in KL divergence gradient estimation for RL
di: Tang, Yunhao, et al.
Pubblicazione: (2025)

RL-finetuning LLMs from on- and off-policy data with a single algorithm
di: Tang, Yunhao, et al.
Pubblicazione: (2025)

Soft Policy Optimization: Online Off-Policy RL for Sequence Models
di: Cohen, Taco, et al.
Pubblicazione: (2025)

An Analysis of Quantile Temporal-Difference Learning
di: Rowland, Mark, et al.
Pubblicazione: (2023)

A Distributional Analogue to the Successor Representation
di: Wiltzer, Harley, et al.
Pubblicazione: (2024)

Understanding the performance gap between online and offline alignment algorithms
di: Tang, Yunhao, et al.
Pubblicazione: (2024)

Generalized Preference Optimization: A Unified Approach to Offline Alignment
di: Tang, Yunhao, et al.
Pubblicazione: (2024)

A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning
di: Khetarpal, Khimya, et al.
Pubblicazione: (2024)

Beyond Verifiable Rewards: Scaling Reinforcement Learning for Language Models to Unverifiable Data
di: Tang, Yunhao, et al.
Pubblicazione: (2025)

Optimizing Language Models for Inference Time Objectives using Reinforcement Learning
di: Tang, Yunhao, et al.
Pubblicazione: (2025)

Human Alignment of Large Language Models through Online Preference Optimisation
di: Calandriello, Daniele, et al.
Pubblicazione: (2024)

DISA: Offline Importance Sampling for Distribution-Matching LLM-RL
di: Wang, Shaobo, et al.
Pubblicazione: (2026)

Efficient RL Training for LLMs with Experience Replay
di: Arnal, Charles, et al.
Pubblicazione: (2026)

Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning
di: Grill, Jean-Bastien, et al.
Pubblicazione: (2026)

Global Optimality and Finite Sample Analysis of Softmax Off-Policy Actor Critic under State Distribution Mismatch
di: Zhang, Shangtong, et al.
Pubblicazione: (2021)

Asymmetric REINFORCE for off-Policy Reinforcement Learning: Balancing positive and negative rewards
di: Arnal, Charles, et al.
Pubblicazione: (2025)

Super-Exponential Regret for UCT, AlphaGo and Variants
di: Orseau, Laurent, et al.
Pubblicazione: (2024)

Low Variance Off-policy Evaluation with State-based Importance Sampling
di: Bossens, David M., et al.
Pubblicazione: (2022)

Distributional Off-policy Evaluation with Bellman Residual Minimization
di: Hong, Sungee, et al.
Pubblicazione: (2024)

Bandits attack function optimization
di: Preux, Philippe, et al.
Pubblicazione: (2026)

Stochastic simultaneous optimistic optimization
di: Valko, Michal, et al.
Pubblicazione: (2026)

Foundations of Multivariate Distributional Reinforcement Learning
di: Wiltzer, Harley, et al.
Pubblicazione: (2024)

Black-box optimization of noisy functions with unknown smoothness
di: Grill, Jean-Bastien, et al.
Pubblicazione: (2026)

Outcome-based Exploration for LLM Reasoning
di: Song, Yuda, et al.
Pubblicazione: (2025)

AIS: Adaptive Importance Sampling for Quantized RL
di: Zhou, Jiajun, et al.
Pubblicazione: (2026)

One-Shot Averaging for Distributed TD($λ$) Under Markov Sampling
di: Tian, Haoxing, et al.
Pubblicazione: (2024)

Patch the Distribution Mismatch: RL Rewriting Agent for Stable Off-Policy SFT
di: Wang, Jiacheng, et al.
Pubblicazione: (2026)

Spectral Thompson sampling
di: Kocak, Tomas, et al.
Pubblicazione: (2026)

Spectral bandits for smooth graph functions
di: Valko, Michal, et al.
Pubblicazione: (2026)

Efficient learning by implicit exploration in bandit problems with side observations
di: Kocak, Tomas, et al.
Pubblicazione: (2026)

Power Distribution Bridges Sampling, Self-Reward RL, and Self-Distillation
di: Tomihari, Akiyoshi, et al.
Pubblicazione: (2026)

SEMDICE: Off-policy State Entropy Maximization via Stationary Distribution Correction Estimation
di: Lee, Jongmin, et al.
Pubblicazione: (2025)

$Q\sharp$: Provably Optimal Distributional RL for LLM Post-Training
di: Zhou, Jin Peng, et al.
Pubblicazione: (2025)

A Finite Sample Complexity Bound for Distributionally Robust Q-learning
di: Wang, Shengbo, et al.
Pubblicazione: (2023)

Distributional Bellman Operators over Mean Embeddings
di: Wenliang, Li Kevin, et al.
Pubblicazione: (2023)

LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LLM Training
di: Wu, Bo, et al.
Pubblicazione: (2025)

Sample Complexity of Variance-reduced Distributionally Robust Q-learning
di: Wang, Shengbo, et al.
Pubblicazione: (2023)