VR-GPT: Visual Language Model for Intelligent Virtual Reality Applications
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909278839767040 |
|---|---|
| author | Konenkov, Mikhail Lykov, Artem Trinitatova, Daria Tsetserukou, Dzmitry |
| author_facet | Konenkov, Mikhail Lykov, Artem Trinitatova, Daria Tsetserukou, Dzmitry |
| contents | The advent of immersive Virtual Reality applications has transformed various domains, yet their integration with advanced artificial intelligence technologies like Visual Language Models remains underexplored. This study introduces a pioneering approach utilizing VLMs within VR environments to enhance user interaction and task efficiency. Leveraging the Unity engine and a custom-developed VLM, our system facilitates real-time, intuitive user interactions through natural language processing, without relying on visual text instructions. The incorporation of speech-to-text and text-to-speech technologies allows for seamless communication between the user and the VLM, enabling the system to guide users through complex tasks effectively. Preliminary experimental results indicate that utilizing VLMs not only reduces task completion times but also improves user comfort and task engagement compared to traditional VR interaction methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_11537 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | VR-GPT: Visual Language Model for Intelligent Virtual Reality Applications Konenkov, Mikhail Lykov, Artem Trinitatova, Daria Tsetserukou, Dzmitry Robotics Artificial Intelligence Emerging Technologies The advent of immersive Virtual Reality applications has transformed various domains, yet their integration with advanced artificial intelligence technologies like Visual Language Models remains underexplored. This study introduces a pioneering approach utilizing VLMs within VR environments to enhance user interaction and task efficiency. Leveraging the Unity engine and a custom-developed VLM, our system facilitates real-time, intuitive user interactions through natural language processing, without relying on visual text instructions. The incorporation of speech-to-text and text-to-speech technologies allows for seamless communication between the user and the VLM, enabling the system to guide users through complex tasks effectively. Preliminary experimental results indicate that utilizing VLMs not only reduces task completion times but also improves user comfort and task engagement compared to traditional VR interaction methods. |
| title | VR-GPT: Visual Language Model for Intelligent Virtual Reality Applications |
| topic | Robotics Artificial Intelligence Emerging Technologies |
| url | https://arxiv.org/abs/2405.11537 |