3D Affordance Keypoint Detection for Robotic Manipulation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911290325204992 |
|---|---|
| author | Liu, Zhiyang Zhao, Ruiteng Zhou, Lei Yuan, Chengran Wu, Yuwei Guo, Sheng Zhang, Zhengshen Liu, Chenchen Ang Jr, Marcelo H |
| author_facet | Liu, Zhiyang Zhao, Ruiteng Zhou, Lei Yuan, Chengran Wu, Yuwei Guo, Sheng Zhang, Zhengshen Liu, Chenchen Ang Jr, Marcelo H |
| contents | This paper presents a novel approach for affordance-informed robotic manipulation by introducing 3D keypoints to enhance the understanding of object parts' functionality. The proposed approach provides direct information about what the potential use of objects is, as well as guidance on where and how a manipulator should engage, whereas conventional methods treat affordance detection as a semantic segmentation task, focusing solely on answering the what question. To address this gap, we propose a Fusion-based Affordance Keypoint Network (FAKP-Net) by introducing 3D keypoint quadruplet that harnesses the synergistic potential of RGB and Depth image to provide information on execution position, direction, and extent. Benchmark testing demonstrates that FAKP-Net outperforms existing models by significant margins in affordance segmentation task and keypoint detection task. Real-world experiments also showcase the reliability of our method in accomplishing manipulation tasks with previously unseen objects. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_22195 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | 3D Affordance Keypoint Detection for Robotic Manipulation Liu, Zhiyang Zhao, Ruiteng Zhou, Lei Yuan, Chengran Wu, Yuwei Guo, Sheng Zhang, Zhengshen Liu, Chenchen Ang Jr, Marcelo H Robotics This paper presents a novel approach for affordance-informed robotic manipulation by introducing 3D keypoints to enhance the understanding of object parts' functionality. The proposed approach provides direct information about what the potential use of objects is, as well as guidance on where and how a manipulator should engage, whereas conventional methods treat affordance detection as a semantic segmentation task, focusing solely on answering the what question. To address this gap, we propose a Fusion-based Affordance Keypoint Network (FAKP-Net) by introducing 3D keypoint quadruplet that harnesses the synergistic potential of RGB and Depth image to provide information on execution position, direction, and extent. Benchmark testing demonstrates that FAKP-Net outperforms existing models by significant margins in affordance segmentation task and keypoint detection task. Real-world experiments also showcase the reliability of our method in accomplishing manipulation tasks with previously unseen objects. |
| title | 3D Affordance Keypoint Detection for Robotic Manipulation |
| topic | Robotics |
| url | https://arxiv.org/abs/2511.22195 |