New Versions of Gradient Temporal Difference Learning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2021
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917570729213952 |
|---|---|
| author | Lee, Donghwan Lim, Han-Dong Park, Jihoon Choi, Okyong |
| author_facet | Lee, Donghwan Lim, Han-Dong Park, Jihoon Choi, Okyong |
| contents | Sutton, Szepesvári and Maei introduced the first gradient temporal-difference (GTD) learning algorithms compatible with both linear function approximation and off-policy training. The goal of this paper is (a) to propose some variants of GTDs with extensive comparative analysis and (b) to establish new theoretical analysis frameworks for the GTDs. These variants are based on convex-concave saddle-point interpretations of GTDs, which effectively unify all the GTDs into a single framework, and provide simple stability analysis based on recent results on primal-dual gradient dynamics. Finally, numerical comparative analysis is given to evaluate these approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2109_04033 |
| institution | arXiv |
| publishDate | 2021 |
| record_format | arxiv |
| spellingShingle | New Versions of Gradient Temporal Difference Learning Lee, Donghwan Lim, Han-Dong Park, Jihoon Choi, Okyong Machine Learning Sutton, Szepesvári and Maei introduced the first gradient temporal-difference (GTD) learning algorithms compatible with both linear function approximation and off-policy training. The goal of this paper is (a) to propose some variants of GTDs with extensive comparative analysis and (b) to establish new theoretical analysis frameworks for the GTDs. These variants are based on convex-concave saddle-point interpretations of GTDs, which effectively unify all the GTDs into a single framework, and provide simple stability analysis based on recent results on primal-dual gradient dynamics. Finally, numerical comparative analysis is given to evaluate these approaches. |
| title | New Versions of Gradient Temporal Difference Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2109.04033 |