Cover Image

A Finite-Time Analysis of TD Learning with Linear Function Approximation without Projections or Strong Convexity

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Wei-Cheng, Orabona, Francesco
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Simple Finite-Time Analysis of TD Learning with Linear Function Approximation
by: Mitra, Aritra
Published: (2024)

A Best-of-Both-Worlds Proof for Tsallis-INF without Fenchel Conjugates
by: Lee, Wei-Cheng, et al.
Published: (2025)

New Lower Bounds for Stochastic Non-Convex Optimization through Divergence Decomposition
by: Saad, El Mehdi, et al.
Published: (2025)

A Modern Introduction to Online Learning
by: Orabona, Francesco
Published: (2019)

A Note on How to Remove the $\ln\ln T$ Term from the Squint Bound
by: Orabona, Francesco
Published: (2026)

New Perspectives on the Polyak Stepsize: Surrogate Functions and Negative Results
by: Orabona, Francesco, et al.
Published: (2025)

SCAFFLSA: Taming Heterogeneity in Federated Linear Stochastic Approximation and TD Learning
by: Mangold, Paul, et al.
Published: (2024)

An Equivalence Between Static and Dynamic Regret Minimization
by: Jacobsen, Andrew, et al.
Published: (2024)

Projection-free Online Learning over Strongly Convex Sets
by: Wan, Yuanyu, et al.
Published: (2020)

Adversarially-Robust TD Learning with Markovian Data: Finite-Time Rates and Fundamental Limits
by: Maity, Sreejeet, et al.
Published: (2025)

Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion
by: Cutkosky, Ashok, et al.
Published: (2023)

Online Learning for Approximately-Convex Functions with Long-term Adversarial Constraints
by: Sarkar, Dhruv, et al.
Published: (2025)

Tight Finite Time Bounds of Two-Time-Scale Linear Stochastic Approximation with Markovian Noise
by: Haque, Shaan Ul, et al.
Published: (2023)

$O(1/k)$ Finite-Time Bound for Non-Linear Two-Time-Scale Stochastic Approximation
by: Chandak, Siddharth
Published: (2025)

Beyond the Ideal: Analyzing the Inexact Muon Update
by: Shulgin, Egor, et al.
Published: (2025)

A primal-dual perspective for distributed TD-learning
by: Lim, Han-Dong, et al.
Published: (2023)

Projection-Free Online Convex Optimization with Time-Varying Constraints
by: Garber, Dan, et al.
Published: (2024)

Heavy-Tailed and Long-Range Dependent Noise in Stochastic Approximation: A Finite-Time Analysis
by: Chandak, Siddharth, et al.
Published: (2026)

Federated Temporal Difference Learning with Linear Function Approximation under Environmental Heterogeneity
by: Wang, Han, et al.
Published: (2023)

Analysis of Off-Policy Multi-Step TD-Learning with Linear Function Approximation
by: Lee, Donghwan
Published: (2024)

Linear Convergence of Entropy-Regularized Natural Policy Gradient with Linear Function Approximation
by: Cayci, Semih, et al.
Published: (2021)

Revisiting Multi-Agent Asynchronous Online Optimization with Delays: the Strongly Convex Case
by: Bao, Lingchan, et al.
Published: (2025)

Finite-Time Decoupled Convergence in Nonlinear Two-Time-Scale Stochastic Approximation
by: Han, Yuze, et al.
Published: (2024)

Reinforcement Learning from Partial Observation: Linear Function Approximation with Provable Sample Efficiency
by: Cai, Qi, et al.
Published: (2022)

Convergence Analysis of the PAGE Stochastic Algorithm for Weakly Convex Finite-Sum Optimization
by: Condat, Laurent, et al.
Published: (2025)

SOC-ICNN: From Polyhedral to Conic Geometry for Learning Convex Surrogate Functions
by: Liu, Kang, et al.
Published: (2026)

Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time
by: Kim, Sungyoon, et al.
Published: (2024)

Infinite-Horizon Reinforcement Learning with Multinomial Logistic Function Approximation
by: Park, Jaehyun, et al.
Published: (2024)

Finite-Time Analysis of On-Policy Heterogeneous Federated Reinforcement Learning
by: Zhang, Chenyu, et al.
Published: (2024)

Provably Efficient Infinite-Horizon Average-Reward Reinforcement Learning with Linear Function Approximation
by: Chae, Woojin, et al.
Published: (2024)

Finite-Time Bounds for Two-Time-Scale Stochastic Approximation with Arbitrary Norm Contractions and Markovian Noise
by: Chandak, Siddharth, et al.
Published: (2025)

ATA: Adaptive Task Allocation for Efficient Resource Management in Distributed Machine Learning
by: Maranjyan, Artavazd, et al.
Published: (2025)

Some Primal-Dual Theory for Subgradient Methods for Strongly Convex Optimization
by: Grimmer, Benjamin, et al.
Published: (2023)

Quantitative Convergence Analysis of Projected Stochastic Gradient Descent for Non-Convex Losses via the Goldstein Subdifferential
by: Zheng, Yuping, et al.
Published: (2025)

$ψ$DAG: Projected Stochastic Approximation Iteration for DAG Structure Learning
by: Ziu, Klea, et al.
Published: (2024)

Universal Online Convex Optimization with $1$ Projection per Round
by: Yang, Wenhao, et al.
Published: (2024)

Gaussian Approximation and Multiplier Bootstrap for Federated Linear Stochastic Approximation
by: Levin, Ilya, et al.
Published: (2026)

Global Optimization of Gaussian Process Acquisition Functions Using a Piecewise-Linear Kernel Approximation
by: Xie, Yilin, et al.
Published: (2024)

Optimistic Safety for Online Convex Optimization with Unknown Linear Constraints
by: Hutchinson, Spencer, et al.
Published: (2024)

Quantizer Design for Finite Model Approximations, Model Learning, and Quantized Q-Learning for MDPs with Unbounded Spaces
by: Bicer, Osman, et al.
Published: (2025)