Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2505.21391 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917012708524032 |
|---|---|
| author | Xie, Zixuan Liu, Xinyu Chandra, Rohan Zhang, Shangtong |
| author_facet | Xie, Zixuan Liu, Xinyu Chandra, Rohan Zhang, Shangtong |
| contents | Linear TD($λ$) is one of the most fundamental reinforcement learning algorithms for policy evaluation. Previously, convergence rates are typically established under the assumption of linearly independent features, which does not hold in many practical scenarios. This paper instead establishes the first $L^2$ convergence rates for linear TD($λ$) operating under arbitrary features, without making any algorithmic modification or additional assumptions. Our results apply to both the discounted and average-reward settings. To address the potential non-uniqueness of solutions resulting from arbitrary features, we develop a novel stochastic approximation result featuring convergence rates to the solution set instead of a single point. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_21391 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Finite Sample Analysis of Linear Temporal Difference Learning with Arbitrary Features Xie, Zixuan Liu, Xinyu Chandra, Rohan Zhang, Shangtong Machine Learning Artificial Intelligence Linear TD($λ$) is one of the most fundamental reinforcement learning algorithms for policy evaluation. Previously, convergence rates are typically established under the assumption of linearly independent features, which does not hold in many practical scenarios. This paper instead establishes the first $L^2$ convergence rates for linear TD($λ$) operating under arbitrary features, without making any algorithmic modification or additional assumptions. Our results apply to both the discounted and average-reward settings. To address the potential non-uniqueness of solutions resulting from arbitrary features, we develop a novel stochastic approximation result featuring convergence rates to the solution set instead of a single point. |
| title | Finite Sample Analysis of Linear Temporal Difference Learning with Arbitrary Features |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2505.21391 |