Mamba Policy: Towards Efficient 3D Diffusion Policy with Hybrid Selective State Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915357844832256 |
|---|---|
| author | Cao, Jiahang Zhang, Qiang Sun, Jingkai Wang, Jiaxu Cheng, Hao Li, Yulin Ma, Jun Wu, Kun Xu, Zhiyuan Shao, Yecheng Zhao, Wen Han, Gang Guo, Yijie Xu, Renjing |
| author_facet | Cao, Jiahang Zhang, Qiang Sun, Jingkai Wang, Jiaxu Cheng, Hao Li, Yulin Ma, Jun Wu, Kun Xu, Zhiyuan Shao, Yecheng Zhao, Wen Han, Gang Guo, Yijie Xu, Renjing |
| contents | Diffusion models have been widely employed in the field of 3D manipulation due to their efficient capability to learn distributions, allowing for precise prediction of action trajectories. However, diffusion models typically rely on large parameter UNet backbones as policy networks, which can be challenging to deploy on resource-constrained devices. Recently, the Mamba model has emerged as a promising solution for efficient modeling, offering low computational complexity and strong performance in sequence modeling. In this work, we propose the Mamba Policy, a lighter but stronger policy that reduces the parameter count by over 80% compared to the original policy network while achieving superior performance. Specifically, we introduce the XMamba Block, which effectively integrates input information with conditional features and leverages a combination of Mamba and Attention mechanisms for deep feature extraction. Extensive experiments demonstrate that the Mamba Policy excels on the Adroit, Dexart, and MetaWorld datasets, requiring significantly fewer computational resources. Additionally, we highlight the Mamba Policy's enhanced robustness in long-horizon scenarios compared to baseline methods and explore the performance of various Mamba variants within the Mamba Policy framework. Real-world experiments are also conducted to further validate its effectiveness. Our open-source project page can be found at https://andycao1125.github.io/mamba_policy/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_07163 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mamba Policy: Towards Efficient 3D Diffusion Policy with Hybrid Selective State Models Cao, Jiahang Zhang, Qiang Sun, Jingkai Wang, Jiaxu Cheng, Hao Li, Yulin Ma, Jun Wu, Kun Xu, Zhiyuan Shao, Yecheng Zhao, Wen Han, Gang Guo, Yijie Xu, Renjing Robotics Computer Vision and Pattern Recognition Diffusion models have been widely employed in the field of 3D manipulation due to their efficient capability to learn distributions, allowing for precise prediction of action trajectories. However, diffusion models typically rely on large parameter UNet backbones as policy networks, which can be challenging to deploy on resource-constrained devices. Recently, the Mamba model has emerged as a promising solution for efficient modeling, offering low computational complexity and strong performance in sequence modeling. In this work, we propose the Mamba Policy, a lighter but stronger policy that reduces the parameter count by over 80% compared to the original policy network while achieving superior performance. Specifically, we introduce the XMamba Block, which effectively integrates input information with conditional features and leverages a combination of Mamba and Attention mechanisms for deep feature extraction. Extensive experiments demonstrate that the Mamba Policy excels on the Adroit, Dexart, and MetaWorld datasets, requiring significantly fewer computational resources. Additionally, we highlight the Mamba Policy's enhanced robustness in long-horizon scenarios compared to baseline methods and explore the performance of various Mamba variants within the Mamba Policy framework. Real-world experiments are also conducted to further validate its effectiveness. Our open-source project page can be found at https://andycao1125.github.io/mamba_policy/. |
| title | Mamba Policy: Towards Efficient 3D Diffusion Policy with Hybrid Selective State Models |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.07163 |