Image de couverture de livre

A Unifying View of Linear Function Approximation in Off-Policy RL Through Matrix Splitting and Preconditioning

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wu, Zechen, Greenwald, Amy, Parr, Ronald
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Approximate Next Policy Sampling: Replacing Conservative Target Policy Updates in Deep RL
par: Sandhu, Dillon, et autres
Publié: (2026)

A Unifying View of Coverage in Linear Off-Policy Evaluation
par: Amortila, Philip, et autres
Publié: (2026)

Analysis of Off-Policy $n$-Step TD-Learning with Linear Function Approximation
par: Lim, Han-Dong, et autres
Publié: (2025)

Analysis of Off-Policy Multi-Step TD-Learning with Linear Function Approximation
par: Lee, Donghwan
Publié: (2024)

Adaptive Layerwise Perturbation: Unifying Off-Policy Corrections for LLM RL
par: Ye, Chenlu, et autres
Publié: (2026)

An Optimal Tightness Bound for the Simulation Lemma
par: Lobel, Sam, et autres
Publié: (2024)

Zero Collapse: A Failure Mode of Policy Gradient Methods in Discontinuous Reward Environments
par: Kumar, Nishant, et autres
Publié: (2026)

Faster Linear Systems and Matrix Norm Approximation via Multi-level Sketched Preconditioning
par: Dereziński, Michał, et autres
Publié: (2024)

Residual Off-Policy RL for Finetuning Behavior Cloning Policies
par: Ankile, Lars, et autres
Publié: (2025)

Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning
par: Zhao, Shiwan, et autres
Publié: (2026)

LLMs Can Learn to Reason Via Off-Policy RL
par: Ritter, Daniel, et autres
Publié: (2026)

Distributionally Robust Off-Dynamics Reinforcement Learning: Provable Efficiency with Linear Function Approximation
par: Liu, Zhishuai, et autres
Publié: (2024)

Policy Learning for Off-Dynamics RL with Deficient Support
par: Van, Linh Le Pham, et autres
Publié: (2024)

Soft Policy Optimization: Online Off-Policy RL for Sequence Models
par: Cohen, Taco, et autres
Publié: (2025)

Provably Efficient RL under Episode-Wise Safety in Constrained MDPs with Linear Function Approximation
par: Kitamura, Toshinori, et autres
Publié: (2025)

On the Nystrom Approximation for Preconditioning in Kernel Machines
par: Abedsoltan, Amirhesam, et autres
Publié: (2023)

PolarGrad: A Class of Matrix-Gradient Optimizers from a Unifying Preconditioning Perspective
par: Lau, Tim Tsz-Kit, et autres
Publié: (2025)

Rethinking the Global Convergence of Softmax Policy Gradient with Linear Function Approximation
par: Lin, Max Qiushi, et autres
Publié: (2025)

Trust the Batch, On- or Off-Policy: Adaptive Policy Optimization for RL Post-Training
par: Fakoor, Rasool, et autres
Publié: (2026)

Kernel Metric Learning for In-Sample Off-Policy Evaluation of Deterministic RL Policies
par: Lee, Haanvid, et autres
Publié: (2024)

Stable Asynchrony: Variance-Controlled Off-Policy RL for LLMs
par: Huang, Luke J., et autres
Publié: (2026)

Linear Convergence of Entropy-Regularized Natural Policy Gradient with Linear Function Approximation
par: Cayci, Semih, et autres
Publié: (2021)

Off-Policy Maximum Entropy RL with Future State and Action Visitation Measures
par: Bolland, Adrien, et autres
Publié: (2024)

RL as Regressor: A Reinforcement Learning Approach for Function Approximation
par: Huang, Yongchao
Publié: (2025)

SOPE: Stabilizing Off-Policy Evaluation for Online RL with Prior Data
par: Romeo, Carlo, et autres
Publié: (2026)

Differentiating Through Integer Linear Programs with Quadratic Regularization and Davis-Yin Splitting
par: McKenzie, Daniel, et autres
Publié: (2023)

Missing Old Logits in Asynchronous Agentic RL: Semantic Mismatch and Repair Methods for Off-Policy Correction
par: Guan, Zhong, et autres
Publié: (2026)

Unifying On- and Off-Policy Variance Reduction Methods
par: Jeunen, Olivier
Publié: (2026)

SCOPE-RL: A Python Library for Offline Reinforcement Learning and Off-Policy Evaluation
par: Kiyohara, Haruka, et autres
Publié: (2023)

Patch the Distribution Mismatch: RL Rewriting Agent for Stable Off-Policy SFT
par: Wang, Jiacheng, et autres
Publié: (2026)

Efficient Recurrent Off-Policy RL Requires a Context-Encoder-Specific Learning Rate
par: Luo, Fan-Ming, et autres
Publié: (2024)

Statistical Inference for Temporal Difference Learning with Linear Function Approximation
par: Wu, Weichen, et autres
Publié: (2024)

Policy Regularized Distributionally Robust Markov Decision Processes with Linear Function Approximation
par: Gu, Jingwen, et autres
Publié: (2025)

OMPO: A Unified Framework for RL under Policy and Dynamics Shifts
par: Luo, Yu, et autres
Publié: (2024)

RMNP: Row-Momentum Normalized Preconditioning for Scalable Matrix-Based Optimization
par: Deng, Shenyang, et autres
Publié: (2026)

NysAct: A Scalable Preconditioned Gradient Descent using Nystrom Approximation
par: Seung, Hyunseok, et autres
Publié: (2025)

Matrix Low-Rank Approximation For Policy Gradient Methods
par: Rozada, Sergio, et autres
Publié: (2024)

MAC: An Efficient Gradient Preconditioning using Mean Activation Approximated Curvature
par: Seung, Hyunseok, et autres
Publié: (2025)

Replicable Reinforcement Learning with Linear Function Approximation
par: Eaton, Eric, et autres
Publié: (2025)

Nonstationary Reinforcement Learning with Linear Function Approximation
par: Zhou, Huozhi, et autres
Publié: (2020)