Language-driven Grasp Detection with Mask-guided Attention
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929440314884096 |
|---|---|
| author | Van Vo, Tuan Vu, Minh Nhat Huang, Baoru Vuong, An Le, Ngan Vo, Thieu Nguyen, Anh |
| author_facet | Van Vo, Tuan Vu, Minh Nhat Huang, Baoru Vuong, An Le, Ngan Vo, Thieu Nguyen, Anh |
| contents | Grasp detection is an essential task in robotics with various industrial applications. However, traditional methods often struggle with occlusions and do not utilize language for grasping. Incorporating natural language into grasp detection remains a challenging task and largely unexplored. To address this gap, we propose a new method for language-driven grasp detection with mask-guided attention by utilizing the transformer attention mechanism with semantic segmentation features. Our approach integrates visual data, segmentation mask features, and natural language instructions, significantly improving grasp detection accuracy. Our work introduces a new framework for language-driven grasp detection, paving the way for language-driven robotic applications. Intensive experiments show that our method outperforms other recent baselines by a clear margin, with a 10.0% success score improvement. We further validate our method in real-world robotic experiments, confirming the effectiveness of our approach. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_19877 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Language-driven Grasp Detection with Mask-guided Attention Van Vo, Tuan Vu, Minh Nhat Huang, Baoru Vuong, An Le, Ngan Vo, Thieu Nguyen, Anh Robotics Computer Vision and Pattern Recognition Grasp detection is an essential task in robotics with various industrial applications. However, traditional methods often struggle with occlusions and do not utilize language for grasping. Incorporating natural language into grasp detection remains a challenging task and largely unexplored. To address this gap, we propose a new method for language-driven grasp detection with mask-guided attention by utilizing the transformer attention mechanism with semantic segmentation features. Our approach integrates visual data, segmentation mask features, and natural language instructions, significantly improving grasp detection accuracy. Our work introduces a new framework for language-driven grasp detection, paving the way for language-driven robotic applications. Intensive experiments show that our method outperforms other recent baselines by a clear margin, with a 10.0% success score improvement. We further validate our method in real-world robotic experiments, confirming the effectiveness of our approach. |
| title | Language-driven Grasp Detection with Mask-guided Attention |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.19877 |