Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2409.14216 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914954552016896 |
|---|---|
| author | Nguyen, Viet Dung Yang, Zhizhuo Buckley, Christopher L. Ororbia, Alexander |
| author_facet | Nguyen, Viet Dung Yang, Zhizhuo Buckley, Christopher L. Ororbia, Alexander |
| contents | Although research has produced promising results demonstrating the utility of active inference (AIF) in Markov decision processes (MDPs), there is relatively less work that builds AIF models in the context of environments and problems that take the form of partially observable Markov decision processes (POMDPs). In POMDP scenarios, the agent must infer the unobserved environmental state from raw sensory observations, e.g., pixels in an image. Additionally, less work exists in examining the most difficult form of POMDP-centered control: continuous action space POMDPs under sparse reward signals. In this work, we address issues facing the AIF modeling paradigm by introducing novel prior preference learning techniques and self-revision schedules to help the agent excel in sparse-reward, continuous action, goal-based robotic control POMDP environments. Empirically, we show that our agents offer improved performance over state-of-the-art models in terms of cumulative rewards, relative stability, and success rate. The code in support of this work can be found at https://github.com/NACLab/robust-active-inference. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_14216 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Models Nguyen, Viet Dung Yang, Zhizhuo Buckley, Christopher L. Ororbia, Alexander Robotics Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning 68T40 (Primary) 68T07, 68T37, 68T05 (Secondary) I.2.9; I.2.10; G.3; I.2.6 Although research has produced promising results demonstrating the utility of active inference (AIF) in Markov decision processes (MDPs), there is relatively less work that builds AIF models in the context of environments and problems that take the form of partially observable Markov decision processes (POMDPs). In POMDP scenarios, the agent must infer the unobserved environmental state from raw sensory observations, e.g., pixels in an image. Additionally, less work exists in examining the most difficult form of POMDP-centered control: continuous action space POMDPs under sparse reward signals. In this work, we address issues facing the AIF modeling paradigm by introducing novel prior preference learning techniques and self-revision schedules to help the agent excel in sparse-reward, continuous action, goal-based robotic control POMDP environments. Empirically, we show that our agents offer improved performance over state-of-the-art models in terms of cumulative rewards, relative stability, and success rate. The code in support of this work can be found at https://github.com/NACLab/robust-active-inference. |
| title | R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Models |
| topic | Robotics Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning 68T40 (Primary) 68T07, 68T37, 68T05 (Secondary) I.2.9; I.2.10; G.3; I.2.6 |
| url | https://arxiv.org/abs/2409.14216 |