Modality-Driven Design for Multi-Step Dexterous Manipulation: Insights from Neuroscience
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915065401180160 |
|---|---|
| author | Wake, Naoki Kanehira, Atsushi Saito, Daichi Takamatsu, Jun Sasabuchi, Kazuhiro Koike, Hideki Ikeuchi, Katsushi |
| author_facet | Wake, Naoki Kanehira, Atsushi Saito, Daichi Takamatsu, Jun Sasabuchi, Kazuhiro Koike, Hideki Ikeuchi, Katsushi |
| contents | Multi-step dexterous manipulation is a fundamental skill in household scenarios, yet remains an underexplored area in robotics. This paper proposes a modular approach, where each step of the manipulation process is addressed with dedicated policies based on effective modality input, rather than relying on a single end-to-end model. To demonstrate this, a dexterous robotic hand performs a manipulation task involving picking up and rotating a box. Guided by insights from neuroscience, the task is decomposed into three sub-skills, 1)reaching, 2)grasping and lifting, and 3)in-hand rotation, based on the dominant sensory modalities employed in the human brain. Each sub-skill is addressed using distinct methods from a practical perspective: a classical controller, a Vision-Language-Action model, and a reinforcement learning policy with force feedback, respectively. We tested the pipeline on a real robot to demonstrate the feasibility of our approach. The key contribution of this study lies in presenting a neuroscience-inspired, modality-driven methodology for multi-step dexterous manipulation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_11337 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Modality-Driven Design for Multi-Step Dexterous Manipulation: Insights from Neuroscience Wake, Naoki Kanehira, Atsushi Saito, Daichi Takamatsu, Jun Sasabuchi, Kazuhiro Koike, Hideki Ikeuchi, Katsushi Robotics Artificial Intelligence Computer Vision and Pattern Recognition Multi-step dexterous manipulation is a fundamental skill in household scenarios, yet remains an underexplored area in robotics. This paper proposes a modular approach, where each step of the manipulation process is addressed with dedicated policies based on effective modality input, rather than relying on a single end-to-end model. To demonstrate this, a dexterous robotic hand performs a manipulation task involving picking up and rotating a box. Guided by insights from neuroscience, the task is decomposed into three sub-skills, 1)reaching, 2)grasping and lifting, and 3)in-hand rotation, based on the dominant sensory modalities employed in the human brain. Each sub-skill is addressed using distinct methods from a practical perspective: a classical controller, a Vision-Language-Action model, and a reinforcement learning policy with force feedback, respectively. We tested the pipeline on a real robot to demonstrate the feasibility of our approach. The key contribution of this study lies in presenting a neuroscience-inspired, modality-driven methodology for multi-step dexterous manipulation. |
| title | Modality-Driven Design for Multi-Step Dexterous Manipulation: Insights from Neuroscience |
| topic | Robotics Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.11337 |