Understanding the theoretical properties of projected Bellman equation, linear Q-learning, and approximate value iteration
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908320338542592 |
|---|---|
| author | Lim, Han-Dong Lee, Donghwan |
| author_facet | Lim, Han-Dong Lee, Donghwan |
| contents | In this paper, we study the theoretical properties of the projected Bellman equation (PBE) and two algorithms to solve this equation: linear Q-learning and approximate value iteration (AVI). We consider two sufficient conditions for the existence of a solution to PBE : strictly negatively row dominating diagonal (SNRDD) assumption and a condition motivated by the convergence of AVI. The SNRDD assumption also ensures the convergence of linear Q-learning, and its relationship with the convergence of AVI is examined. Lastly, several interesting observations on the solution of PBE are provided when using $ε$-greedy policy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_10865 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Understanding the theoretical properties of projected Bellman equation, linear Q-learning, and approximate value iteration Lim, Han-Dong Lee, Donghwan Artificial Intelligence Machine Learning In this paper, we study the theoretical properties of the projected Bellman equation (PBE) and two algorithms to solve this equation: linear Q-learning and approximate value iteration (AVI). We consider two sufficient conditions for the existence of a solution to PBE : strictly negatively row dominating diagonal (SNRDD) assumption and a condition motivated by the convergence of AVI. The SNRDD assumption also ensures the convergence of linear Q-learning, and its relationship with the convergence of AVI is examined. Lastly, several interesting observations on the solution of PBE are provided when using $ε$-greedy policy. |
| title | Understanding the theoretical properties of projected Bellman equation, linear Q-learning, and approximate value iteration |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2504.10865 |