Efficient Imitation Without Demonstrations via Value-Penalized Auxiliary Control from Examples
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911151663611904 |
|---|---|
| author | Ablett, Trevor Chan, Bryan Wang, Jayce Haoran Kelly, Jonathan |
| author_facet | Ablett, Trevor Chan, Bryan Wang, Jayce Haoran Kelly, Jonathan |
| contents | Common approaches to providing feedback in reinforcement learning are the use of hand-crafted rewards or full-trajectory expert demonstrations. Alternatively, one can use examples of completed tasks, but such an approach can be extremely sample inefficient. We introduce value-penalized auxiliary control from examples (VPACE), an algorithm that significantly improves exploration in example-based control by adding examples of simple auxiliary tasks and an above-success-level value penalty. Across both simulated and real robotic environments, we show that our approach substantially improves learning efficiency for challenging tasks, while maintaining bounded value estimates. Preliminary results also suggest that VPACE may learn more efficiently than the more common approaches of using full trajectories or true sparse rewards. Project site: https://papers.starslab.ca/vpace/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_03311 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Efficient Imitation Without Demonstrations via Value-Penalized Auxiliary Control from Examples Ablett, Trevor Chan, Bryan Wang, Jayce Haoran Kelly, Jonathan Robotics Artificial Intelligence Machine Learning Common approaches to providing feedback in reinforcement learning are the use of hand-crafted rewards or full-trajectory expert demonstrations. Alternatively, one can use examples of completed tasks, but such an approach can be extremely sample inefficient. We introduce value-penalized auxiliary control from examples (VPACE), an algorithm that significantly improves exploration in example-based control by adding examples of simple auxiliary tasks and an above-success-level value penalty. Across both simulated and real robotic environments, we show that our approach substantially improves learning efficiency for challenging tasks, while maintaining bounded value estimates. Preliminary results also suggest that VPACE may learn more efficiently than the more common approaches of using full trajectories or true sparse rewards. Project site: https://papers.starslab.ca/vpace/. |
| title | Efficient Imitation Without Demonstrations via Value-Penalized Auxiliary Control from Examples |
| topic | Robotics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2407.03311 |