Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2505.02216 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916731344125952 |
|---|---|
| author | Curtis, Aidan Tang, Hao Veloso, Thiago Ellis, Kevin Tenenbaum, Joshua Lozano-Pérez, Tomás Kaelbling, Leslie Pack |
| author_facet | Curtis, Aidan Tang, Hao Veloso, Thiago Ellis, Kevin Tenenbaum, Joshua Lozano-Pérez, Tomás Kaelbling, Leslie Pack |
| contents | Partially Observable Markov Decision Processes (POMDPs) model decision making under uncertainty. While there are many approaches to approximately solving POMDPs, we aim to address the problem of learning such models. In particular, we are interested in a subclass of POMDPs wherein the components of the model, including the observation function, reward function, transition function, and initial state distribution function, can be modeled as low-complexity probabilistic graphical models in the form of a short probabilistic program. Our strategy to learn these programs uses an LLM as a prior, generating candidate probabilistic programs that are then tested against the empirical distribution and adjusted through feedback. We experiment on a number of classical toy POMDP problems, simulated MiniGrid domains, and two real mobile-base robotics search domains involving partial observability. Our results show that using an LLM to guide in the construction of a low-complexity POMDP model can be more effective than tabular POMDP learning, behavior cloning, or direct LLM planning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_02216 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LLM-Guided Probabilistic Program Induction for POMDP Model Estimation Curtis, Aidan Tang, Hao Veloso, Thiago Ellis, Kevin Tenenbaum, Joshua Lozano-Pérez, Tomás Kaelbling, Leslie Pack Artificial Intelligence Partially Observable Markov Decision Processes (POMDPs) model decision making under uncertainty. While there are many approaches to approximately solving POMDPs, we aim to address the problem of learning such models. In particular, we are interested in a subclass of POMDPs wherein the components of the model, including the observation function, reward function, transition function, and initial state distribution function, can be modeled as low-complexity probabilistic graphical models in the form of a short probabilistic program. Our strategy to learn these programs uses an LLM as a prior, generating candidate probabilistic programs that are then tested against the empirical distribution and adjusted through feedback. We experiment on a number of classical toy POMDP problems, simulated MiniGrid domains, and two real mobile-base robotics search domains involving partial observability. Our results show that using an LLM to guide in the construction of a low-complexity POMDP model can be more effective than tabular POMDP learning, behavior cloning, or direct LLM planning. |
| title | LLM-Guided Probabilistic Program Induction for POMDP Model Estimation |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2505.02216 |