MoRE: Unlocking Scalability in Reinforcement Learning for Quadruped Vision-Language-Action Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909533703503872 |
|---|---|
| author | Zhao, Han Song, Wenxuan Wang, Donglin Tong, Xinyang Ding, Pengxiang Cheng, Xuelian Ge, Zongyuan |
| author_facet | Zhao, Han Song, Wenxuan Wang, Donglin Tong, Xinyang Ding, Pengxiang Cheng, Xuelian Ge, Zongyuan |
| contents | Developing versatile quadruped robots that can smoothly perform various actions and tasks in real-world environments remains a significant challenge. This paper introduces a novel vision-language-action (VLA) model, mixture of robotic experts (MoRE), for quadruped robots that aim to introduce reinforcement learning (RL) for fine-tuning large-scale VLA models with a large amount of mixed-quality data. MoRE integrates multiple low-rank adaptation modules as distinct experts within a dense multi-modal large language model (MLLM), forming a sparse-activated mixture-of-experts model. This design enables the model to effectively adapt to a wide array of downstream tasks. Moreover, we employ a reinforcement learning-based training objective to train our model as a Q-function after deeply exploring the structural properties of our tasks. Effective learning from automatically collected mixed-quality data enhances data efficiency and model performance. Extensive experiments demonstrate that MoRE outperforms all baselines across six different skills and exhibits superior generalization capabilities in out-of-distribution scenarios. We further validate our method in real-world scenarios, confirming the practicality of our approach and laying a solid foundation for future research on multi-task learning in quadruped robots. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_08007 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MoRE: Unlocking Scalability in Reinforcement Learning for Quadruped Vision-Language-Action Models Zhao, Han Song, Wenxuan Wang, Donglin Tong, Xinyang Ding, Pengxiang Cheng, Xuelian Ge, Zongyuan Robotics Artificial Intelligence Developing versatile quadruped robots that can smoothly perform various actions and tasks in real-world environments remains a significant challenge. This paper introduces a novel vision-language-action (VLA) model, mixture of robotic experts (MoRE), for quadruped robots that aim to introduce reinforcement learning (RL) for fine-tuning large-scale VLA models with a large amount of mixed-quality data. MoRE integrates multiple low-rank adaptation modules as distinct experts within a dense multi-modal large language model (MLLM), forming a sparse-activated mixture-of-experts model. This design enables the model to effectively adapt to a wide array of downstream tasks. Moreover, we employ a reinforcement learning-based training objective to train our model as a Q-function after deeply exploring the structural properties of our tasks. Effective learning from automatically collected mixed-quality data enhances data efficiency and model performance. Extensive experiments demonstrate that MoRE outperforms all baselines across six different skills and exhibits superior generalization capabilities in out-of-distribution scenarios. We further validate our method in real-world scenarios, confirming the practicality of our approach and laying a solid foundation for future research on multi-task learning in quadruped robots. |
| title | MoRE: Unlocking Scalability in Reinforcement Learning for Quadruped Vision-Language-Action Models |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2503.08007 |