Learning to Walk with Less: a Dyna-Style Approach to Quadrupedal Locomotion
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908524535087104 |
|---|---|
| author | Affonso, Francisco Tommaselli, Felipe Andrade G. Negri, Juliano Medeiros, Vivian S. Gasparino, Mateus V. Chowdhary, Girish Becker, Marcelo |
| author_facet | Affonso, Francisco Tommaselli, Felipe Andrade G. Negri, Juliano Medeiros, Vivian S. Gasparino, Mateus V. Chowdhary, Girish Becker, Marcelo |
| contents | Traditional RL-based locomotion controllers often suffer from low data efficiency, requiring extensive interaction to achieve robust performance. We present a model-based reinforcement learning (MBRL) framework that improves sample efficiency for quadrupedal locomotion by appending synthetic data to the end of standard rollouts in PPO-based controllers, following the Dyna-Style paradigm. A predictive model, trained alongside the policy, generates short-horizon synthetic transitions that are gradually integrated using a scheduling strategy based on the policy update iterations. Through an ablation study, we identified a strong correlation between sample efficiency and rollout length, which guided the design of our experiments. We validated our approach in simulation on the Unitree Go1 robot and showed that replacing part of the simulated steps with synthetic ones not only mimics extended rollouts but also improves policy return and reduces variance. Finally, we demonstrate that this improvement transfers to the ability to track a wide range of locomotion commands using fewer simulated steps. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_06296 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Learning to Walk with Less: a Dyna-Style Approach to Quadrupedal Locomotion Affonso, Francisco Tommaselli, Felipe Andrade G. Negri, Juliano Medeiros, Vivian S. Gasparino, Mateus V. Chowdhary, Girish Becker, Marcelo Robotics Artificial Intelligence Traditional RL-based locomotion controllers often suffer from low data efficiency, requiring extensive interaction to achieve robust performance. We present a model-based reinforcement learning (MBRL) framework that improves sample efficiency for quadrupedal locomotion by appending synthetic data to the end of standard rollouts in PPO-based controllers, following the Dyna-Style paradigm. A predictive model, trained alongside the policy, generates short-horizon synthetic transitions that are gradually integrated using a scheduling strategy based on the policy update iterations. Through an ablation study, we identified a strong correlation between sample efficiency and rollout length, which guided the design of our experiments. We validated our approach in simulation on the Unitree Go1 robot and showed that replacing part of the simulated steps with synthetic ones not only mimics extended rollouts but also improves policy return and reduces variance. Finally, we demonstrate that this improvement transfers to the ability to track a wide range of locomotion commands using fewer simulated steps. |
| title | Learning to Walk with Less: a Dyna-Style Approach to Quadrupedal Locomotion |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2509.06296 |