Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910369963835392 |
|---|---|
| author | Zhao, Mingde Alver, Safa van Seijen, Harm Laroche, Romain Precup, Doina Bengio, Yoshua |
| author_facet | Zhao, Mingde Alver, Safa van Seijen, Harm Laroche, Romain Precup, Doina Bengio, Yoshua |
| contents | Inspired by human conscious planning, we propose Skipper, a model-based reinforcement learning framework utilizing spatio-temporal abstractions to generalize better in novel situations. It automatically decomposes the given task into smaller, more manageable subtasks, and thus enables sparse decision-making and focused computation on the relevant parts of the environment. The decomposition relies on the extraction of an abstracted proxy problem represented as a directed graph, in which vertices and edges are learned end-to-end from hindsight. Our theoretical analyses provide performance guarantees under appropriate assumptions and establish where our approach is expected to be helpful. Generalization-focused experiments validate Skipper's significant advantage in zero-shot generalization, compared to some existing state-of-the-art hierarchical planning methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_00229 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learning Zhao, Mingde Alver, Safa van Seijen, Harm Laroche, Romain Precup, Doina Bengio, Yoshua Artificial Intelligence Machine Learning Inspired by human conscious planning, we propose Skipper, a model-based reinforcement learning framework utilizing spatio-temporal abstractions to generalize better in novel situations. It automatically decomposes the given task into smaller, more manageable subtasks, and thus enables sparse decision-making and focused computation on the relevant parts of the environment. The decomposition relies on the extraction of an abstracted proxy problem represented as a directed graph, in which vertices and edges are learned end-to-end from hindsight. Our theoretical analyses provide performance guarantees under appropriate assumptions and establish where our approach is expected to be helpful. Generalization-focused experiments validate Skipper's significant advantage in zero-shot generalization, compared to some existing state-of-the-art hierarchical planning methods. |
| title | Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learning |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2310.00229 |