When in Doubt, Think Slow: Iterative Reasoning with Latent Imagination
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910341915475968 |
|---|---|
| author | Benfeghoul, Martin Zahid, Umais Guo, Qinghai Fountas, Zafeirios |
| author_facet | Benfeghoul, Martin Zahid, Umais Guo, Qinghai Fountas, Zafeirios |
| contents | In an unfamiliar setting, a model-based reinforcement learning agent can be limited by the accuracy of its world model. In this work, we present a novel, training-free approach to improving the performance of such agents separately from planning and learning. We do so by applying iterative inference at decision-time, to fine-tune the inferred agent states based on the coherence of future state representations. Our approach achieves a consistent improvement in both reconstruction accuracy and task performance when applied to visual 3D navigation tasks. We go on to show that considering more future states further improves the performance of the agent in partially-observable environments, but not in a fully-observable one. Finally, we demonstrate that agents with less training pre-evaluation benefit most from our approach. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_15283 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | When in Doubt, Think Slow: Iterative Reasoning with Latent Imagination Benfeghoul, Martin Zahid, Umais Guo, Qinghai Fountas, Zafeirios Machine Learning Artificial Intelligence I.2.0; I.2.8; I.2.10; I.4.5; I.4.10 In an unfamiliar setting, a model-based reinforcement learning agent can be limited by the accuracy of its world model. In this work, we present a novel, training-free approach to improving the performance of such agents separately from planning and learning. We do so by applying iterative inference at decision-time, to fine-tune the inferred agent states based on the coherence of future state representations. Our approach achieves a consistent improvement in both reconstruction accuracy and task performance when applied to visual 3D navigation tasks. We go on to show that considering more future states further improves the performance of the agent in partially-observable environments, but not in a fully-observable one. Finally, we demonstrate that agents with less training pre-evaluation benefit most from our approach. |
| title | When in Doubt, Think Slow: Iterative Reasoning with Latent Imagination |
| topic | Machine Learning Artificial Intelligence I.2.0; I.2.8; I.2.10; I.4.5; I.4.10 |
| url | https://arxiv.org/abs/2402.15283 |