InteRACT: Transformer Models for Human Intent Prediction Conditioned on Robot Actions
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916270205566976 |
|---|---|
| author | Kedia, Kushal Bhardwaj, Atiksh Dan, Prithwish Choudhury, Sanjiban |
| author_facet | Kedia, Kushal Bhardwaj, Atiksh Dan, Prithwish Choudhury, Sanjiban |
| contents | In collaborative human-robot manipulation, a robot must predict human intents and adapt its actions accordingly to smoothly execute tasks. However, the human's intent in turn depends on actions the robot takes, creating a chicken-or-egg problem. Prior methods ignore such inter-dependency and instead train marginal intent prediction models independent of robot actions. This is because training conditional models is hard given a lack of paired human-robot interaction datasets. Can we instead leverage large-scale human-human interaction data that is more easily accessible? Our key insight is to exploit a correspondence between human and robot actions that enables transfer learning from human-human to human-robot data. We propose a novel architecture, InteRACT, that pre-trains a conditional intent prediction model on large human-human datasets and fine-tunes on a small human-robot dataset. We evaluate on a set of real-world collaborative human-robot manipulation tasks and show that our conditional model improves over various marginal baselines. We also introduce new techniques to tele-operate a 7-DoF robot arm and collect a diverse range of human-robot collaborative manipulation data, which we open-source. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_12943 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | InteRACT: Transformer Models for Human Intent Prediction Conditioned on Robot Actions Kedia, Kushal Bhardwaj, Atiksh Dan, Prithwish Choudhury, Sanjiban Robotics Artificial Intelligence Machine Learning Multiagent Systems In collaborative human-robot manipulation, a robot must predict human intents and adapt its actions accordingly to smoothly execute tasks. However, the human's intent in turn depends on actions the robot takes, creating a chicken-or-egg problem. Prior methods ignore such inter-dependency and instead train marginal intent prediction models independent of robot actions. This is because training conditional models is hard given a lack of paired human-robot interaction datasets. Can we instead leverage large-scale human-human interaction data that is more easily accessible? Our key insight is to exploit a correspondence between human and robot actions that enables transfer learning from human-human to human-robot data. We propose a novel architecture, InteRACT, that pre-trains a conditional intent prediction model on large human-human datasets and fine-tunes on a small human-robot dataset. We evaluate on a set of real-world collaborative human-robot manipulation tasks and show that our conditional model improves over various marginal baselines. We also introduce new techniques to tele-operate a 7-DoF robot arm and collect a diverse range of human-robot collaborative manipulation data, which we open-source. |
| title | InteRACT: Transformer Models for Human Intent Prediction Conditioned on Robot Actions |
| topic | Robotics Artificial Intelligence Machine Learning Multiagent Systems |
| url | https://arxiv.org/abs/2311.12943 |