Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.11923 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909080420876288 |
|---|---|
| author | Wang, Zhan Yuan, Lin-Ping Wang, Liangwei Jiang, Bingchuan Zeng, Wei |
| author_facet | Wang, Zhan Yuan, Lin-Ping Wang, Liangwei Jiang, Bingchuan Zeng, Wei |
| contents | Tour guidance in virtual museums encourages multi-modal interactions to boost user experiences, concerning engagement, immersion, and spatial awareness. Nevertheless, achieving the goal is challenging due to the complexity of comprehending diverse user needs and accommodating personalized user preferences. Informed by a formative study that characterizes guidance-seeking contexts, we establish a multi-modal interaction design framework for virtual tour guidance. We then design VirtuWander, a two-stage innovative system using domain-oriented large language models to transform user inquiries into diverse guidance-seeking contexts and facilitate multi-modal interactions. The feasibility and versatility of VirtuWander are demonstrated with virtual guiding examples that encompass various touring scenarios and cater to personalized preferences. We further evaluate VirtuWander through a user study within an immersive simulated museum. The results suggest that our system enhances engaging virtual tour experiences through personalized communication and knowledgeable assistance, indicating its potential for expanding into real-world scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_11923 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | VirtuWander: Enhancing Multi-modal Interaction for Virtual Tour Guidance through Large Language Models Wang, Zhan Yuan, Lin-Ping Wang, Liangwei Jiang, Bingchuan Zeng, Wei Human-Computer Interaction Tour guidance in virtual museums encourages multi-modal interactions to boost user experiences, concerning engagement, immersion, and spatial awareness. Nevertheless, achieving the goal is challenging due to the complexity of comprehending diverse user needs and accommodating personalized user preferences. Informed by a formative study that characterizes guidance-seeking contexts, we establish a multi-modal interaction design framework for virtual tour guidance. We then design VirtuWander, a two-stage innovative system using domain-oriented large language models to transform user inquiries into diverse guidance-seeking contexts and facilitate multi-modal interactions. The feasibility and versatility of VirtuWander are demonstrated with virtual guiding examples that encompass various touring scenarios and cater to personalized preferences. We further evaluate VirtuWander through a user study within an immersive simulated museum. The results suggest that our system enhances engaging virtual tour experiences through personalized communication and knowledgeable assistance, indicating its potential for expanding into real-world scenarios. |
| title | VirtuWander: Enhancing Multi-modal Interaction for Virtual Tour Guidance through Large Language Models |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2401.11923 |