On Bellman equations for continuous-time policy evaluation I: discretization and approximation
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929413947392000 |
|---|---|
| author | Mou, Wenlong Zhu, Yuhua |
| author_facet | Mou, Wenlong Zhu, Yuhua |
| contents | We study the problem of computing the value function from a discretely-observed trajectory of a continuous-time diffusion process. We develop a new class of algorithms based on easily implementable numerical schemes that are compatible with discrete-time reinforcement learning (RL) with function approximation. We establish high-order numerical accuracy as well as the approximation error guarantees for the proposed approach. In contrast to discrete-time RL problems where the approximation factor depends on the effective horizon, we obtain a bounded approximation factor using the underlying elliptic structures, even if the effective horizon diverges to infinity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_05966 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | On Bellman equations for continuous-time policy evaluation I: discretization and approximation Mou, Wenlong Zhu, Yuhua Machine Learning Numerical Analysis Optimization and Control Probability We study the problem of computing the value function from a discretely-observed trajectory of a continuous-time diffusion process. We develop a new class of algorithms based on easily implementable numerical schemes that are compatible with discrete-time reinforcement learning (RL) with function approximation. We establish high-order numerical accuracy as well as the approximation error guarantees for the proposed approach. In contrast to discrete-time RL problems where the approximation factor depends on the effective horizon, we obtain a bounded approximation factor using the underlying elliptic structures, even if the effective horizon diverges to infinity. |
| title | On Bellman equations for continuous-time policy evaluation I: discretization and approximation |
| topic | Machine Learning Numerical Analysis Optimization and Control Probability |
| url | https://arxiv.org/abs/2407.05966 |