Average Cost Optimality of Partially Observed MDPS: Contraction of Non-linear Filters, Optimal Solutions and Approximations
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914893392773120 |
|---|---|
| author | Demirci, Yunus Emre Kara, Ali Devran Yüksel, Serdar |
| author_facet | Demirci, Yunus Emre Kara, Ali Devran Yüksel, Serdar |
| contents | The average cost optimality is known to be a challenging problem for partially observable stochastic control, with few results available beyond the finite state, action, and measurement setup, for which somewhat restrictive conditions are available. In this paper, we present explicit and easily testable conditions for the existence of solutions to the average cost optimality equation where the state space is compact. In particular, we present a new contraction based analysis, which is new to the literature to our knowledge, building on recent regularity results for non-linear filters. Beyond establishing existence, we also present several implications of our analysis that are new to the literature: (i) robustness to incorrect priors (ii) near optimality of policies based on quantized approximations, (iii) near optimality of policies with finite memory, and (iv) convergence in Q-learning. In addition to our main theorem, each of these represents a novel contribution for average cost criteria. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_14111 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Average Cost Optimality of Partially Observed MDPS: Contraction of Non-linear Filters, Optimal Solutions and Approximations Demirci, Yunus Emre Kara, Ali Devran Yüksel, Serdar Optimization and Control 90C40, 93E11, 93E20 The average cost optimality is known to be a challenging problem for partially observable stochastic control, with few results available beyond the finite state, action, and measurement setup, for which somewhat restrictive conditions are available. In this paper, we present explicit and easily testable conditions for the existence of solutions to the average cost optimality equation where the state space is compact. In particular, we present a new contraction based analysis, which is new to the literature to our knowledge, building on recent regularity results for non-linear filters. Beyond establishing existence, we also present several implications of our analysis that are new to the literature: (i) robustness to incorrect priors (ii) near optimality of policies based on quantized approximations, (iii) near optimality of policies with finite memory, and (iv) convergence in Q-learning. In addition to our main theorem, each of these represents a novel contribution for average cost criteria. |
| title | Average Cost Optimality of Partially Observed MDPS: Contraction of Non-linear Filters, Optimal Solutions and Approximations |
| topic | Optimization and Control 90C40, 93E11, 93E20 |
| url | https://arxiv.org/abs/2312.14111 |