Addressing Myopic Constrained POMDP Planning with Recursive Dual Ascent
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917622169206784 |
|---|---|
| author | Stocco, Paula Chundi, Suhas Jamgochian, Arec Kochenderfer, Mykel J. |
| author_facet | Stocco, Paula Chundi, Suhas Jamgochian, Arec Kochenderfer, Mykel J. |
| contents | Lagrangian-guided Monte Carlo tree search with global dual ascent has been applied to solve large constrained partially observable Markov decision processes (CPOMDPs) online. In this work, we demonstrate that these global dual parameters can lead to myopic action selection during exploration, ultimately leading to suboptimal decision making. To address this, we introduce history-dependent dual variables that guide local action selection and are optimized with recursive dual ascent. We empirically compare the performance of our approach on a motivating toy example and two large CPOMDPs, demonstrating improved exploration, and ultimately, safer outcomes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_17358 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Addressing Myopic Constrained POMDP Planning with Recursive Dual Ascent Stocco, Paula Chundi, Suhas Jamgochian, Arec Kochenderfer, Mykel J. Artificial Intelligence Lagrangian-guided Monte Carlo tree search with global dual ascent has been applied to solve large constrained partially observable Markov decision processes (CPOMDPs) online. In this work, we demonstrate that these global dual parameters can lead to myopic action selection during exploration, ultimately leading to suboptimal decision making. To address this, we introduce history-dependent dual variables that guide local action selection and are optimized with recursive dual ascent. We empirically compare the performance of our approach on a motivating toy example and two large CPOMDPs, demonstrating improved exploration, and ultimately, safer outcomes. |
| title | Addressing Myopic Constrained POMDP Planning with Recursive Dual Ascent |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2403.17358 |