Point & Grasp: Flexible Selection of Out-of-Reach Objects Through Probabilistic Cue Integration
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914504697184256 |
|---|---|
| author | Luo, Xuejing Moon, Hee-Seung Holz, Christian Oulasvirta, Antti |
| author_facet | Luo, Xuejing Moon, Hee-Seung Holz, Christian Oulasvirta, Antti |
| contents | Selecting out-of-reach objects is a fundamental task in mixed reality (MR). Existing methods rely on a single cue or deterministically fuse multiple cues, leading to performance degradation when the dominant cue becomes unreliable. In this work, we introduce a probabilistic cue integration framework that enables flexible combination of multiple user-generated cues for intent inference. Inspired by natural grasping behavior, we instantiate the framework with pointing direction and grasp gestures as a new interaction technique, Point&Grasp. To this end, we collect the Out-of-Reach Grasping (ORG) dataset to train a robust likelihood model of the gestural cue, which captures grasping patterns not present in existing in-reach datasets. User studies demonstrate that our selection method with cue integration not only improves accuracy and speed over single-cue baselines, but also remains practically effective compared to state-of-the-art methods across various sources of ambiguity. The dataset and code are available at https://github.com/drlxj/point-and-grasp. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_22491 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Point & Grasp: Flexible Selection of Out-of-Reach Objects Through Probabilistic Cue Integration Luo, Xuejing Moon, Hee-Seung Holz, Christian Oulasvirta, Antti Human-Computer Interaction Robotics Selecting out-of-reach objects is a fundamental task in mixed reality (MR). Existing methods rely on a single cue or deterministically fuse multiple cues, leading to performance degradation when the dominant cue becomes unreliable. In this work, we introduce a probabilistic cue integration framework that enables flexible combination of multiple user-generated cues for intent inference. Inspired by natural grasping behavior, we instantiate the framework with pointing direction and grasp gestures as a new interaction technique, Point&Grasp. To this end, we collect the Out-of-Reach Grasping (ORG) dataset to train a robust likelihood model of the gestural cue, which captures grasping patterns not present in existing in-reach datasets. User studies demonstrate that our selection method with cue integration not only improves accuracy and speed over single-cue baselines, but also remains practically effective compared to state-of-the-art methods across various sources of ambiguity. The dataset and code are available at https://github.com/drlxj/point-and-grasp. |
| title | Point & Grasp: Flexible Selection of Out-of-Reach Objects Through Probabilistic Cue Integration |
| topic | Human-Computer Interaction Robotics |
| url | https://arxiv.org/abs/2604.22491 |