Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2403.01962 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909138959728640 |
|---|---|
| author | Shi, Haojie Li, Tingguang Zhu, Qingxu Sheng, Jiapeng Han, Lei Meng, Max Q. -H. |
| author_facet | Shi, Haojie Li, Tingguang Zhu, Qingxu Sheng, Jiapeng Han, Lei Meng, Max Q. -H. |
| contents | Learning-based methods have improved locomotion skills of quadruped robots through deep reinforcement learning. However, the sim-to-real gap and low sample efficiency still limit the skill transfer. To address this issue, we propose an efficient model-based learning framework that combines a world model with a policy network. We train a differentiable world model to predict future states and use it to directly supervise a Variational Autoencoder (VAE)-based policy network to imitate real animal behaviors. This significantly reduces the need for real interaction data and allows for rapid policy updates. We also develop a high-level network to track diverse commands and trajectories. Our simulated results show a tenfold sample efficiency increase compared to reinforcement learning methods such as PPO. In real-world testing, our policy achieves proficient command-following performance with only a two-minute data collection period and generalizes well to new speeds and paths. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_01962 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | An Efficient Model-Based Approach on Learning Agile Motor Skills without Reinforcement Shi, Haojie Li, Tingguang Zhu, Qingxu Sheng, Jiapeng Han, Lei Meng, Max Q. -H. Robotics Learning-based methods have improved locomotion skills of quadruped robots through deep reinforcement learning. However, the sim-to-real gap and low sample efficiency still limit the skill transfer. To address this issue, we propose an efficient model-based learning framework that combines a world model with a policy network. We train a differentiable world model to predict future states and use it to directly supervise a Variational Autoencoder (VAE)-based policy network to imitate real animal behaviors. This significantly reduces the need for real interaction data and allows for rapid policy updates. We also develop a high-level network to track diverse commands and trajectories. Our simulated results show a tenfold sample efficiency increase compared to reinforcement learning methods such as PPO. In real-world testing, our policy achieves proficient command-following performance with only a two-minute data collection period and generalizes well to new speeds and paths. |
| title | An Efficient Model-Based Approach on Learning Agile Motor Skills without Reinforcement |
| topic | Robotics |
| url | https://arxiv.org/abs/2403.01962 |