RAGNet: Large-scale Reasoning-based Affordance Segmentation Benchmark towards General Grasping
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918108752510976 |
|---|---|
| author | Wu, Dongming Fu, Yanping Huang, Saike Liu, Yingfei Jia, Fan Liu, Nian Dai, Feng Wang, Tiancai Anwer, Rao Muhammad Khan, Fahad Shahbaz Shen, Jianbing |
| author_facet | Wu, Dongming Fu, Yanping Huang, Saike Liu, Yingfei Jia, Fan Liu, Nian Dai, Feng Wang, Tiancai Anwer, Rao Muhammad Khan, Fahad Shahbaz Shen, Jianbing |
| contents | General robotic grasping systems require accurate object affordance perception in diverse open-world scenarios following human instructions. However, current studies suffer from the problem of lacking reasoning-based large-scale affordance prediction data, leading to considerable concern about open-world effectiveness. To address this limitation, we build a large-scale grasping-oriented affordance segmentation benchmark with human-like instructions, named RAGNet. It contains 273k images, 180 categories, and 26k reasoning instructions. The images cover diverse embodied data domains, such as wild, robot, ego-centric, and even simulation data. They are carefully annotated with an affordance map, while the difficulty of language instructions is largely increased by removing their category name and only providing functional descriptions. Furthermore, we propose a comprehensive affordance-based grasping framework, named AffordanceNet, which consists of a VLM pre-trained on our massive affordance data and a grasping network that conditions an affordance map to grasp the target. Extensive experiments on affordance segmentation benchmarks and real-robot manipulation tasks show that our model has a powerful open-world generalization ability. Our data and code is available at https://github.com/wudongming97/AffordanceNet. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_23734 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RAGNet: Large-scale Reasoning-based Affordance Segmentation Benchmark towards General Grasping Wu, Dongming Fu, Yanping Huang, Saike Liu, Yingfei Jia, Fan Liu, Nian Dai, Feng Wang, Tiancai Anwer, Rao Muhammad Khan, Fahad Shahbaz Shen, Jianbing Computer Vision and Pattern Recognition Robotics General robotic grasping systems require accurate object affordance perception in diverse open-world scenarios following human instructions. However, current studies suffer from the problem of lacking reasoning-based large-scale affordance prediction data, leading to considerable concern about open-world effectiveness. To address this limitation, we build a large-scale grasping-oriented affordance segmentation benchmark with human-like instructions, named RAGNet. It contains 273k images, 180 categories, and 26k reasoning instructions. The images cover diverse embodied data domains, such as wild, robot, ego-centric, and even simulation data. They are carefully annotated with an affordance map, while the difficulty of language instructions is largely increased by removing their category name and only providing functional descriptions. Furthermore, we propose a comprehensive affordance-based grasping framework, named AffordanceNet, which consists of a VLM pre-trained on our massive affordance data and a grasping network that conditions an affordance map to grasp the target. Extensive experiments on affordance segmentation benchmarks and real-robot manipulation tasks show that our model has a powerful open-world generalization ability. Our data and code is available at https://github.com/wudongming97/AffordanceNet. |
| title | RAGNet: Large-scale Reasoning-based Affordance Segmentation Benchmark towards General Grasping |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2507.23734 |