Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.29419 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910088385527808 |
|---|---|
| author | Zhuang, Qiyuan Xu, He-Yang Wang, Yijun Zhao, Xin-Yang Li, Yang-Yang Wei, Xiu-Shen |
| author_facet | Zhuang, Qiyuan Xu, He-Yang Wang, Yijun Zhao, Xin-Yang Li, Yang-Yang Wei, Xiu-Shen |
| contents | Understanding object affordances is essential for enabling robots to perform purposeful and fine-grained interactions in diverse and unstructured environments. However, existing approaches either rely on retrieval, which is fragile due to sparsity and coverage gaps, or on large-scale models, which frequently mislocalize contact points and mispredict post-contact actions when applied to unseen categories, thereby hindering robust generalization. We introduce Retrieval-Augmented Affordance Prediction (RAAP), a framework that unifies affordance retrieval with alignment-based learning. By decoupling static contact localization and dynamic action direction, RAAP transfers contact points via dense correspondence and predicts action directions through a retrieval-augmented alignment model that consolidates multiple references with dual-weighted attention. Trained on compact subsets of DROID and HOI4D with as few as tens of samples per task, RAAP achieves consistent performance across unseen objects and categories, and enables zero-shot robotic manipulation in both simulation and the real world. Project website: https://github.com/SEU-VIPGroup/RAAP. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_29419 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | RAAP: Retrieval-Augmented Affordance Prediction with Cross-Image Action Alignment Zhuang, Qiyuan Xu, He-Yang Wang, Yijun Zhao, Xin-Yang Li, Yang-Yang Wei, Xiu-Shen Robotics Artificial Intelligence Computer Vision and Pattern Recognition Understanding object affordances is essential for enabling robots to perform purposeful and fine-grained interactions in diverse and unstructured environments. However, existing approaches either rely on retrieval, which is fragile due to sparsity and coverage gaps, or on large-scale models, which frequently mislocalize contact points and mispredict post-contact actions when applied to unseen categories, thereby hindering robust generalization. We introduce Retrieval-Augmented Affordance Prediction (RAAP), a framework that unifies affordance retrieval with alignment-based learning. By decoupling static contact localization and dynamic action direction, RAAP transfers contact points via dense correspondence and predicts action directions through a retrieval-augmented alignment model that consolidates multiple references with dual-weighted attention. Trained on compact subsets of DROID and HOI4D with as few as tens of samples per task, RAAP achieves consistent performance across unseen objects and categories, and enables zero-shot robotic manipulation in both simulation and the real world. Project website: https://github.com/SEU-VIPGroup/RAAP. |
| title | RAAP: Retrieval-Augmented Affordance Prediction with Cross-Image Action Alignment |
| topic | Robotics Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.29419 |