M3Bench: Benchmarking Whole-body Motion Generation for Mobile Manipulation in 3D Scenes
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908383147196416 |
|---|---|
| author | Zhang, Zeyu Yan, Sixu Han, Muzhi Wang, Zaijin Wang, Xinggang Zhu, Song-Chun Liu, Hangxin |
| author_facet | Zhang, Zeyu Yan, Sixu Han, Muzhi Wang, Zaijin Wang, Xinggang Zhu, Song-Chun Liu, Hangxin |
| contents | We propose M3Bench, a new benchmark for whole-body motion generation in mobile manipulation tasks. Given a 3D scene context, M3Bench requires an embodied agent to reason about its configuration, environmental constraints, and task objectives to generate coordinated whole-body motion trajectories for object rearrangement. M3Bench features 30,000 object rearrangement tasks across 119 diverse scenes, providing expert demonstrations generated by our newly developed M3BenchMaker, an automatic data generation tool that produces whole-body motion trajectories from high-level task instructions using only basic scene and robot information. Our benchmark includes various task splits to evaluate generalization across different dimensions and leverages realistic physics simulation for trajectory assessment. Extensive evaluation analysis reveals that state-of-the-art models struggle with coordinating base-arm motion while adhering to environmental and task-specific constraints, underscoring the need for new models to bridge this gap. By releasing M3Bench and M3BenchMaker we aim to advance robotics research toward more adaptive and capable mobile manipulation in diverse, real-world environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_06678 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | M3Bench: Benchmarking Whole-body Motion Generation for Mobile Manipulation in 3D Scenes Zhang, Zeyu Yan, Sixu Han, Muzhi Wang, Zaijin Wang, Xinggang Zhu, Song-Chun Liu, Hangxin Robotics Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning We propose M3Bench, a new benchmark for whole-body motion generation in mobile manipulation tasks. Given a 3D scene context, M3Bench requires an embodied agent to reason about its configuration, environmental constraints, and task objectives to generate coordinated whole-body motion trajectories for object rearrangement. M3Bench features 30,000 object rearrangement tasks across 119 diverse scenes, providing expert demonstrations generated by our newly developed M3BenchMaker, an automatic data generation tool that produces whole-body motion trajectories from high-level task instructions using only basic scene and robot information. Our benchmark includes various task splits to evaluate generalization across different dimensions and leverages realistic physics simulation for trajectory assessment. Extensive evaluation analysis reveals that state-of-the-art models struggle with coordinating base-arm motion while adhering to environmental and task-specific constraints, underscoring the need for new models to bridge this gap. By releasing M3Bench and M3BenchMaker we aim to advance robotics research toward more adaptive and capable mobile manipulation in diverse, real-world environments. |
| title | M3Bench: Benchmarking Whole-body Motion Generation for Mobile Manipulation in 3D Scenes |
| topic | Robotics Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2410.06678 |