Ego-Exo 3D Hand Tracking in the Wild with a Mobile Multi-Camera Rig
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911190510206976 |
|---|---|
| author | Rim, Patrick He, Kun Harris, Kevin Copple, Braden Han, Shangchen An, Sizhe Shugurov, Ivan Hodan, Tomas Wen, He Xie, Xu |
| author_facet | Rim, Patrick He, Kun Harris, Kevin Copple, Braden Han, Shangchen An, Sizhe Shugurov, Ivan Hodan, Tomas Wen, He Xie, Xu |
| contents | Accurate 3D tracking of hands and their interactions with the world in unconstrained settings remains a significant challenge for egocentric computer vision. With few exceptions, existing datasets are predominantly captured in controlled lab setups, limiting environmental diversity and model generalization. To address this, we introduce a novel marker-less multi-camera system designed to capture precise 3D hands and objects, which allows for nearly unconstrained mobility in genuinely in-the-wild conditions. We combine a lightweight, back-mounted capture rig with eight exocentric cameras, and a user-worn Meta Quest 3 headset, which contributes two egocentric views. We design an ego-exo tracking pipeline to generate accurate 3D hand pose ground truth from this system, and rigorously evaluate its quality. By collecting an annotated dataset featuring synchronized multi-view images and precise 3D hand poses, we demonstrate the capability of our approach to significantly reduce the trade-off between environmental realism and 3D annotation accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_02601 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Ego-Exo 3D Hand Tracking in the Wild with a Mobile Multi-Camera Rig Rim, Patrick He, Kun Harris, Kevin Copple, Braden Han, Shangchen An, Sizhe Shugurov, Ivan Hodan, Tomas Wen, He Xie, Xu Computer Vision and Pattern Recognition Accurate 3D tracking of hands and their interactions with the world in unconstrained settings remains a significant challenge for egocentric computer vision. With few exceptions, existing datasets are predominantly captured in controlled lab setups, limiting environmental diversity and model generalization. To address this, we introduce a novel marker-less multi-camera system designed to capture precise 3D hands and objects, which allows for nearly unconstrained mobility in genuinely in-the-wild conditions. We combine a lightweight, back-mounted capture rig with eight exocentric cameras, and a user-worn Meta Quest 3 headset, which contributes two egocentric views. We design an ego-exo tracking pipeline to generate accurate 3D hand pose ground truth from this system, and rigorously evaluate its quality. By collecting an annotated dataset featuring synchronized multi-view images and precise 3D hand poses, we demonstrate the capability of our approach to significantly reduce the trade-off between environmental realism and 3D annotation accuracy. |
| title | Ego-Exo 3D Hand Tracking in the Wild with a Mobile Multi-Camera Rig |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.02601 |