Cover Image

Convergence of Two-Timescale Markovian Stochastic Approximations with Applications in Reinforcement Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Mahadevan, Vagul, Chen, Claire, Liu, Shuze Daniel, Zhang, Shangtong
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The ODE Method for Stochastic Approximation and Reinforcement Learning with Markovian Noise
by: Liu, Shuze Daniel, et al.
Published: (2024)

Doubly Optimal Policy Evaluation for Reinforcement Learning
by: Liu, Shuze Daniel, et al.
Published: (2024)

Efficient Multi-Policy Evaluation for Reinforcement Learning
by: Liu, Shuze Daniel, et al.
Published: (2024)

Efficient Policy Evaluation with Safety Constraint for Reinforcement Learning
by: Chen, Claire, et al.
Published: (2024)

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise
by: Qian, Xiaochi, et al.
Published: (2024)

Efficient Policy Evaluation with Offline Data Informed Behavior Policy Design
by: Liu, Shuze, et al.
Published: (2023)

Beyond Linear Attention: Softmax Transformers Implement In-Context Reinforcement Learning
by: Xie, Zixuan, et al.
Published: (2026)

Almost Sure Convergence Rates of Stochastic Approximation and Reinforcement Learning via a Poisson-Moreau Drift
by: Liu, Xinyu, et al.
Published: (2026)

Asymptotic and Finite Sample Analysis of Nonexpansive Stochastic Approximations with Markovian Noise
by: Blaser, Ethan, et al.
Published: (2024)

Central Limit Theorem for Two-Timescale Stochastic Approximation with Markovian Noise: Theory and Applications
by: Hu, Jie, et al.
Published: (2024)

Predicting Plasticity in Deep Continual Learning: A Theoretical Perspective
by: Wang, Jiuqi, et al.
Published: (2026)

GameChat: Multi-LLM Dialogue for Safe, Agile, and Socially Optimal Multi-Agent Navigation in Constrained Environments
by: Mahadevan, Vagul, et al.
Published: (2025)

Convergence and Emergence of In-Context Reinforcement Learning with Chain of Thought
by: Xie, Zixuan, et al.
Published: (2026)

Towards Formalizing Reinforcement Learning Theory
by: Zhang, Shangtong
Published: (2025)

Extensions of Robbins-Siegmund Theorem with Applications in Reinforcement Learning
by: Liu, Xinyu, et al.
Published: (2025)

Almost Sure Convergence of Linear Temporal Difference Learning with Arbitrary Features
by: Wang, Jiuqi, et al.
Published: (2024)

Universal Reinforcement Learning in Coalgebras: Asynchronous Stochastic Computation via Conduction
by: Mahadevan, Sridhar
Published: (2025)

Gaussian Approximation for Two-Timescale Linear Stochastic Approximation
by: Butyrin, Bogdan, et al.
Published: (2025)

MathlibLemma: Folklore Lemma Generation and Benchmark for Formal Mathematics
by: Liu, Xinyu, et al.
Published: (2026)

Linear $Q$-Learning Does Not Diverge in $L^2$: Convergence Rates to a Bounded Set
by: Liu, Xinyu, et al.
Published: (2025)

Two-Timescale Linear Stochastic Approximation: Constant Stepsizes Go a Long Way
by: Kwon, Jeongyeol, et al.
Published: (2024)

Towards Provable Emergence of In-Context Reinforcement Learning
by: Wang, Jiuqi, et al.
Published: (2025)

Offline Two-Player Zero-Sum Markov Games with KL Regularization
by: Chen, Claire, et al.
Published: (2026)

Single-Timescale Multi-Sequence Stochastic Approximation Without Fixed Point Smoothness: Theories and Applications
by: Huang, Yue, et al.
Published: (2024)

Almost Sure Convergence of Differential Temporal Difference Learning for Average Reward Markov Decision Processes
by: Blaser, Ethan, et al.
Published: (2026)

Counterfactual Explanations for Continuous Action Reinforcement Learning
by: Dong, Shuyang, et al.
Published: (2025)

Convergence of Batch Asynchronous Stochastic Approximation With Applications to Reinforcement Learning
by: Karandikar, Rajeeva L., et al.
Published: (2021)

CRASH: Challenging Reinforcement-Learning Based Adversarial Scenarios For Safety Hardening
by: Kulkarni, Amar, et al.
Published: (2024)

Convergence Rate in Nonlinear Two-Time-Scale Stochastic Approximation with State (Time)-Dependence
by: Chen, Zixi, et al.
Published: (2025)

Transformers Can Learn Temporal Difference Methods for In-Context Reinforcement Learning
by: Wang, Jiuqi, et al.
Published: (2024)

Two-Timescale Critic-Actor for Average Reward MDPs with Function Approximation
by: Panda, Prashansa, et al.
Published: (2024)

Finite-Time Decoupled Convergence in Nonlinear Two-Time-Scale Stochastic Approximation
by: Han, Yuze, et al.
Published: (2024)

On the Divergence of Differential Temporal Difference Learning without Local Clocks
by: Antrobius, David, et al.
Published: (2026)

Revisiting a Design Choice in Gradient Temporal Difference Learning
by: Qian, Xiaochi, et al.
Published: (2023)

On the Convergence of Single-Timescale Actor-Critic
by: Kumar, Navdeep, et al.
Published: (2024)

Bias and Extrapolation in Markovian Linear Stochastic Approximation with Constant Stepsizes
by: Huo, Dongyan, et al.
Published: (2022)

Group Fairness in Multi-Task Reinforcement Learning
by: Song, Kefan, et al.
Published: (2025)

Federated Stochastic Approximation under Markov Noise and Heterogeneity: Applications in Reinforcement Learning
by: Khodadadian, Sajad, et al.
Published: (2022)

Convergence of Two Time-Scale Stochastic Approximation: A Martingale Approach
by: Vidyasagar, Mathukumalli
Published: (2026)

Multi-agent Markov Entanglement
by: Chen, Shuze, et al.
Published: (2025)