Constrained Skill Discovery: Quadruped Locomotion with Unsupervised Reinforcement Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912067565387776 |
|---|---|
| author | Atanassov, Vassil Yu, Wanming Mitchell, Alexander Luis Finean, Mark Nicholas Havoutis, Ioannis |
| author_facet | Atanassov, Vassil Yu, Wanming Mitchell, Alexander Luis Finean, Mark Nicholas Havoutis, Ioannis |
| contents | Representation learning and unsupervised skill discovery can allow robots to acquire diverse and reusable behaviors without the need for task-specific rewards. In this work, we use unsupervised reinforcement learning to learn a latent representation by maximizing the mutual information between skills and states subject to a distance constraint. Our method improves upon prior constrained skill discovery methods by replacing the latent transition maximization with a norm-matching objective. This not only results in a much a richer state space coverage compared to baseline methods, but allows the robot to learn more stable and easily controllable locomotive behaviors. We successfully deploy the learned policy on a real ANYmal quadruped robot and demonstrate that the robot can accurately reach arbitrary points of the Cartesian state space in a zero-shot manner, using only an intrinsic skill discovery and standard regularization rewards. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_07877 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Constrained Skill Discovery: Quadruped Locomotion with Unsupervised Reinforcement Learning Atanassov, Vassil Yu, Wanming Mitchell, Alexander Luis Finean, Mark Nicholas Havoutis, Ioannis Robotics Representation learning and unsupervised skill discovery can allow robots to acquire diverse and reusable behaviors without the need for task-specific rewards. In this work, we use unsupervised reinforcement learning to learn a latent representation by maximizing the mutual information between skills and states subject to a distance constraint. Our method improves upon prior constrained skill discovery methods by replacing the latent transition maximization with a norm-matching objective. This not only results in a much a richer state space coverage compared to baseline methods, but allows the robot to learn more stable and easily controllable locomotive behaviors. We successfully deploy the learned policy on a real ANYmal quadruped robot and demonstrate that the robot can accurately reach arbitrary points of the Cartesian state space in a zero-shot manner, using only an intrinsic skill discovery and standard regularization rewards. |
| title | Constrained Skill Discovery: Quadruped Locomotion with Unsupervised Reinforcement Learning |
| topic | Robotics |
| url | https://arxiv.org/abs/2410.07877 |