TacUMI: A Multi-Modal Universal Manipulation Interface for Contact-Rich Tasks
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911388842065920 |
|---|---|
| author | Cheng, Tailai Chen, Kejia Chen, Lingyun Zhang, Liding Zhang, Yue Ling, Yao Hamad, Mahdi Bing, Zhenshan Wu, Fan Sharma, Karan Knoll, Alois |
| author_facet | Cheng, Tailai Chen, Kejia Chen, Lingyun Zhang, Liding Zhang, Yue Ling, Yao Hamad, Mahdi Bing, Zhenshan Wu, Fan Sharma, Karan Knoll, Alois |
| contents | Task decomposition is critical for understanding and learning complex long-horizon manipulation tasks. Especially for tasks involving rich physical interactions, relying solely on visual observations and robot proprioceptive information often fails to reveal the underlying event transitions. This raises the requirement for efficient collection of high-quality multi-modal data as well as robust segmentation method to decompose demonstrations into meaningful modules. Building on the idea of the handheld demonstration device Universal Manipulation Interface (UMI), we introduce TacUMI, a multi-modal data collection system that integrates additionally ViTac sensors, force-torque sensor, and pose tracker into a compact, robot-compatible gripper design, which enables synchronized acquisition of all these modalities during human demonstrations. We then propose a multi-modal segmentation framework that leverages temporal models to detect semantically meaningful event boundaries in sequential manipulations. Evaluation on a challenging cable mounting task shows more than 90 percent segmentation accuracy and highlights a remarkable improvement with more modalities, which validates that TacUMI establishes a practical foundation for both scalable collection and segmentation of multi-modal demonstrations in contact-rich tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_14550 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TacUMI: A Multi-Modal Universal Manipulation Interface for Contact-Rich Tasks Cheng, Tailai Chen, Kejia Chen, Lingyun Zhang, Liding Zhang, Yue Ling, Yao Hamad, Mahdi Bing, Zhenshan Wu, Fan Sharma, Karan Knoll, Alois Robotics Task decomposition is critical for understanding and learning complex long-horizon manipulation tasks. Especially for tasks involving rich physical interactions, relying solely on visual observations and robot proprioceptive information often fails to reveal the underlying event transitions. This raises the requirement for efficient collection of high-quality multi-modal data as well as robust segmentation method to decompose demonstrations into meaningful modules. Building on the idea of the handheld demonstration device Universal Manipulation Interface (UMI), we introduce TacUMI, a multi-modal data collection system that integrates additionally ViTac sensors, force-torque sensor, and pose tracker into a compact, robot-compatible gripper design, which enables synchronized acquisition of all these modalities during human demonstrations. We then propose a multi-modal segmentation framework that leverages temporal models to detect semantically meaningful event boundaries in sequential manipulations. Evaluation on a challenging cable mounting task shows more than 90 percent segmentation accuracy and highlights a remarkable improvement with more modalities, which validates that TacUMI establishes a practical foundation for both scalable collection and segmentation of multi-modal demonstrations in contact-rich tasks. |
| title | TacUMI: A Multi-Modal Universal Manipulation Interface for Contact-Rich Tasks |
| topic | Robotics |
| url | https://arxiv.org/abs/2601.14550 |