Modality Selection and Skill Segmentation via Cross-Modality Attention
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912337765597184 |
|---|---|
| author | Jiang, Jiawei Ota, Kei Jha, Devesh K. Kanezaki, Asako |
| author_facet | Jiang, Jiawei Ota, Kei Jha, Devesh K. Kanezaki, Asako |
| contents | Incorporating additional sensory modalities such as tactile and audio into foundational robotic models poses significant challenges due to the curse of dimensionality. This work addresses this issue through modality selection. We propose a cross-modality attention (CMA) mechanism to identify and selectively utilize the modalities that are most informative for action generation at each timestep. Furthermore, we extend the application of CMA to segment primitive skills from expert demonstrations and leverage this segmentation to train a hierarchical policy capable of solving long-horizon, contact-rich manipulation tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_14573 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Modality Selection and Skill Segmentation via Cross-Modality Attention Jiang, Jiawei Ota, Kei Jha, Devesh K. Kanezaki, Asako Robotics Artificial Intelligence Incorporating additional sensory modalities such as tactile and audio into foundational robotic models poses significant challenges due to the curse of dimensionality. This work addresses this issue through modality selection. We propose a cross-modality attention (CMA) mechanism to identify and selectively utilize the modalities that are most informative for action generation at each timestep. Furthermore, we extend the application of CMA to segment primitive skills from expert demonstrations and leverage this segmentation to train a hierarchical policy capable of solving long-horizon, contact-rich manipulation tasks. |
| title | Modality Selection and Skill Segmentation via Cross-Modality Attention |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2504.14573 |