Taming "data-hungry" reinforcement learning? Stability in continuous state-action spaces
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929206133260288 |
|---|---|
| author | Duan, Yaqi Wainwright, Martin J. |
| author_facet | Duan, Yaqi Wainwright, Martin J. |
| contents | We introduce a novel framework for analyzing reinforcement learning (RL) in continuous state-action spaces, and use it to prove fast rates of convergence in both off-line and on-line settings. Our analysis highlights two key stability properties, relating to how changes in value functions and/or policies affect the Bellman operator and occupation measures. We argue that these properties are satisfied in many continuous state-action Markov decision processes, and demonstrate how they arise naturally when using linear function approximation methods. Our analysis offers fresh perspectives on the roles of pessimism and optimism in off-line and on-line RL, and highlights the connection between off-line RL and transfer learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_05233 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Taming "data-hungry" reinforcement learning? Stability in continuous state-action spaces Duan, Yaqi Wainwright, Martin J. Machine Learning Information Theory Systems and Control Optimization and Control We introduce a novel framework for analyzing reinforcement learning (RL) in continuous state-action spaces, and use it to prove fast rates of convergence in both off-line and on-line settings. Our analysis highlights two key stability properties, relating to how changes in value functions and/or policies affect the Bellman operator and occupation measures. We argue that these properties are satisfied in many continuous state-action Markov decision processes, and demonstrate how they arise naturally when using linear function approximation methods. Our analysis offers fresh perspectives on the roles of pessimism and optimism in off-line and on-line RL, and highlights the connection between off-line RL and transfer learning. |
| title | Taming "data-hungry" reinforcement learning? Stability in continuous state-action spaces |
| topic | Machine Learning Information Theory Systems and Control Optimization and Control |
| url | https://arxiv.org/abs/2401.05233 |