Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2311.03534 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910480461725696 |
|---|---|
| author | Koul, Anurag Sujit, Shivakanth Chen, Shaoru Evans, Ben Wu, Lili Xu, Byron Chari, Rajan Islam, Riashat Seraj, Raihan Efroni, Yonathan Molu, Lekan Dudik, Miro Langford, John Lamb, Alex |
| author_facet | Koul, Anurag Sujit, Shivakanth Chen, Shaoru Evans, Ben Wu, Lili Xu, Byron Chari, Rajan Islam, Riashat Seraj, Raihan Efroni, Yonathan Molu, Lekan Dudik, Miro Langford, John Lamb, Alex |
| contents | Goal-conditioned planning benefits from learned low-dimensional representations of rich observations. While compact latent representations typically learned from variational autoencoders or inverse dynamics enable goal-conditioned decision making, they ignore state reachability, hampering their performance. In this paper, we learn a representation that associates reachable states together for effective planning and goal-conditioned policy learning. We first learn a latent representation with multi-step inverse dynamics (to remove distracting information), and then transform this representation to associate reachable states together in $\ell_2$ space. Our proposals are rigorously tested in various simulation testbeds. Numerical results in reward-based settings show significant improvements in sampling efficiency. Further, in reward-free settings this approach yields layered state abstractions that enable computationally efficient hierarchical planning for reaching ad hoc goals with zero additional samples. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_03534 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | PcLast: Discovering Plannable Continuous Latent States Koul, Anurag Sujit, Shivakanth Chen, Shaoru Evans, Ben Wu, Lili Xu, Byron Chari, Rajan Islam, Riashat Seraj, Raihan Efroni, Yonathan Molu, Lekan Dudik, Miro Langford, John Lamb, Alex Machine Learning Artificial Intelligence Robotics Goal-conditioned planning benefits from learned low-dimensional representations of rich observations. While compact latent representations typically learned from variational autoencoders or inverse dynamics enable goal-conditioned decision making, they ignore state reachability, hampering their performance. In this paper, we learn a representation that associates reachable states together for effective planning and goal-conditioned policy learning. We first learn a latent representation with multi-step inverse dynamics (to remove distracting information), and then transform this representation to associate reachable states together in $\ell_2$ space. Our proposals are rigorously tested in various simulation testbeds. Numerical results in reward-based settings show significant improvements in sampling efficiency. Further, in reward-free settings this approach yields layered state abstractions that enable computationally efficient hierarchical planning for reaching ad hoc goals with zero additional samples. |
| title | PcLast: Discovering Plannable Continuous Latent States |
| topic | Machine Learning Artificial Intelligence Robotics |
| url | https://arxiv.org/abs/2311.03534 |