Pessimistic Iterative Planning with RNNs for Robust POMDPs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911121236033536 |
|---|---|
| author | Galesloot, Maris F. L. Suilen, Marnix Simão, Thiago D. Carr, Steven Spaan, Matthijs T. J. Topcu, Ufuk Jansen, Nils |
| author_facet | Galesloot, Maris F. L. Suilen, Marnix Simão, Thiago D. Carr, Steven Spaan, Matthijs T. J. Topcu, Ufuk Jansen, Nils |
| contents | Robust POMDPs extend classical POMDPs to incorporate model uncertainty using so-called uncertainty sets on the transition and observation functions, effectively defining ranges of probabilities. Policies for robust POMDPs must be (1) memory-based to account for partial observability and (2) robust against model uncertainty to account for the worst-case probability instances from the uncertainty sets. To compute such robust memory-based policies, we propose the pessimistic iterative planning (PIP) framework, which alternates between (1) selecting pessimistic POMDPs via worst-case probability instances from the uncertainty sets, and (2) computing finite-state controllers (FSCs) for these pessimistic POMDPs. Within PIP, we propose the rFSCNet algorithm, which optimizes a recurrent neural network to compute the FSCs. The empirical evaluation shows that rFSCNet can compute better-performing robust policies than several baselines and a state-of-the-art robust POMDP solver. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_08770 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Pessimistic Iterative Planning with RNNs for Robust POMDPs Galesloot, Maris F. L. Suilen, Marnix Simão, Thiago D. Carr, Steven Spaan, Matthijs T. J. Topcu, Ufuk Jansen, Nils Artificial Intelligence Machine Learning Robust POMDPs extend classical POMDPs to incorporate model uncertainty using so-called uncertainty sets on the transition and observation functions, effectively defining ranges of probabilities. Policies for robust POMDPs must be (1) memory-based to account for partial observability and (2) robust against model uncertainty to account for the worst-case probability instances from the uncertainty sets. To compute such robust memory-based policies, we propose the pessimistic iterative planning (PIP) framework, which alternates between (1) selecting pessimistic POMDPs via worst-case probability instances from the uncertainty sets, and (2) computing finite-state controllers (FSCs) for these pessimistic POMDPs. Within PIP, we propose the rFSCNet algorithm, which optimizes a recurrent neural network to compute the FSCs. The empirical evaluation shows that rFSCNet can compute better-performing robust policies than several baselines and a state-of-the-art robust POMDP solver. |
| title | Pessimistic Iterative Planning with RNNs for Robust POMDPs |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2408.08770 |