Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2408.06481 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917973338357760 |
|---|---|
| author | Xu, Zhengtong Uppuluri, Raghava Zhang, Xinwei Fitch, Cael Crandall, Philip Glen Shou, Wan Wang, Dongyi She, Yu |
| author_facet | Xu, Zhengtong Uppuluri, Raghava Zhang, Xinwei Fitch, Cael Crandall, Philip Glen Shou, Wan Wang, Dongyi She, Yu |
| contents | UniT is an approach to tactile representation learning, using VQGAN to learn a compact latent space and serve as the tactile representation. It uses tactile images obtained from a single simple object to train the representation with generalizability. This tactile representation can be zero-shot transferred to various downstream tasks, including perception tasks and manipulation policy learning. Our benchmarkings on in-hand 3D pose and 6D pose estimation tasks and a tactile classification task show that UniT outperforms existing visual and tactile representation learning methods. Additionally, UniT's effectiveness in policy learning is demonstrated across three real-world tasks involving diverse manipulated objects and complex robot-object-environment interactions. Through extensive experimentation, UniT is shown to be a simple-to-train, plug-and-play, yet widely effective method for tactile representation learning. For more details, please refer to our open-source repository https://github.com/ZhengtongXu/UniT and the project website https://zhengtongxu.github.io/unit-website/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_06481 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | UniT: Data Efficient Tactile Representation with Generalization to Unseen Objects Xu, Zhengtong Uppuluri, Raghava Zhang, Xinwei Fitch, Cael Crandall, Philip Glen Shou, Wan Wang, Dongyi She, Yu Robotics UniT is an approach to tactile representation learning, using VQGAN to learn a compact latent space and serve as the tactile representation. It uses tactile images obtained from a single simple object to train the representation with generalizability. This tactile representation can be zero-shot transferred to various downstream tasks, including perception tasks and manipulation policy learning. Our benchmarkings on in-hand 3D pose and 6D pose estimation tasks and a tactile classification task show that UniT outperforms existing visual and tactile representation learning methods. Additionally, UniT's effectiveness in policy learning is demonstrated across three real-world tasks involving diverse manipulated objects and complex robot-object-environment interactions. Through extensive experimentation, UniT is shown to be a simple-to-train, plug-and-play, yet widely effective method for tactile representation learning. For more details, please refer to our open-source repository https://github.com/ZhengtongXu/UniT and the project website https://zhengtongxu.github.io/unit-website/. |
| title | UniT: Data Efficient Tactile Representation with Generalization to Unseen Objects |
| topic | Robotics |
| url | https://arxiv.org/abs/2408.06481 |