OCTO+: A Suite for Automatic Open-Vocabulary Object Placement in Mixed Reality
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909075666632704 |
|---|---|
| author | Sharma, Aditya Yoffe, Luke Höllerer, Tobias |
| author_facet | Sharma, Aditya Yoffe, Luke Höllerer, Tobias |
| contents | One key challenge in Augmented Reality is the placement of virtual content in natural locations. Most existing automated techniques can only work with a closed-vocabulary, fixed set of objects. In this paper, we introduce and evaluate several methods for automatic object placement using recent advances in open-vocabulary vision-language models. Through a multifaceted evaluation, we identify a new state-of-the-art method, OCTO+. We also introduce a benchmark for automatically evaluating the placement of virtual objects in augmented reality, alleviating the need for costly user studies. Through this, in addition to human evaluations, we find that OCTO+ places objects in a valid region over 70% of the time, outperforming other methods on a range of metrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_08973 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | OCTO+: A Suite for Automatic Open-Vocabulary Object Placement in Mixed Reality Sharma, Aditya Yoffe, Luke Höllerer, Tobias Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language One key challenge in Augmented Reality is the placement of virtual content in natural locations. Most existing automated techniques can only work with a closed-vocabulary, fixed set of objects. In this paper, we introduce and evaluate several methods for automatic object placement using recent advances in open-vocabulary vision-language models. Through a multifaceted evaluation, we identify a new state-of-the-art method, OCTO+. We also introduce a benchmark for automatically evaluating the placement of virtual objects in augmented reality, alleviating the need for costly user studies. Through this, in addition to human evaluations, we find that OCTO+ places objects in a valid region over 70% of the time, outperforming other methods on a range of metrics. |
| title | OCTO+: A Suite for Automatic Open-Vocabulary Object Placement in Mixed Reality |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2401.08973 |