ImaginateAR: AI-Assisted In-Situ Authoring in Augmented Reality
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914166839705600 |
|---|---|
| author | Lee, Jaewook Aleotti, Filippo Mazala, Diego Garcia-Hernando, Guillermo Vicente, Sara Johnston, Oliver James Kraus-Liang, Isabel Powierza, Jakub Shin, Donghoon Froehlich, Jon E. Brostow, Gabriel Van Brummelen, Jessica |
| author_facet | Lee, Jaewook Aleotti, Filippo Mazala, Diego Garcia-Hernando, Guillermo Vicente, Sara Johnston, Oliver James Kraus-Liang, Isabel Powierza, Jakub Shin, Donghoon Froehlich, Jon E. Brostow, Gabriel Van Brummelen, Jessica |
| contents | While augmented reality (AR) enables new ways to play, tell stories, and explore ideas rooted in the physical world, authoring personalized AR content remains difficult for non-experts, often requiring professional tools and time. Prior systems have explored AI-driven XR design but typically rely on manually defined VR environments and fixed asset libraries, limiting creative flexibility and real-world relevance. We introduce ImaginateAR, the first mobile tool for AI-assisted AR authoring to combine offline scene understanding, fast 3D asset generation, and LLMs -- enabling users to create outdoor scenes through natural language interaction. For example, saying "a dragon enjoying a campfire" (P7) prompts the system to generate and arrange relevant assets, which can then be refined manually. Our technical evaluation shows that our custom pipelines produce more accurate outdoor scene graphs and generate 3D meshes faster than prior methods. A three-part user study (N=20) revealed preferred roles for AI, how users create in freeform use, and design implications for future AR authoring tools. ImaginateAR takes a step toward empowering anyone to create AR experiences anywhere -- simply by speaking their imagination. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_21360 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ImaginateAR: AI-Assisted In-Situ Authoring in Augmented Reality Lee, Jaewook Aleotti, Filippo Mazala, Diego Garcia-Hernando, Guillermo Vicente, Sara Johnston, Oliver James Kraus-Liang, Isabel Powierza, Jakub Shin, Donghoon Froehlich, Jon E. Brostow, Gabriel Van Brummelen, Jessica Human-Computer Interaction While augmented reality (AR) enables new ways to play, tell stories, and explore ideas rooted in the physical world, authoring personalized AR content remains difficult for non-experts, often requiring professional tools and time. Prior systems have explored AI-driven XR design but typically rely on manually defined VR environments and fixed asset libraries, limiting creative flexibility and real-world relevance. We introduce ImaginateAR, the first mobile tool for AI-assisted AR authoring to combine offline scene understanding, fast 3D asset generation, and LLMs -- enabling users to create outdoor scenes through natural language interaction. For example, saying "a dragon enjoying a campfire" (P7) prompts the system to generate and arrange relevant assets, which can then be refined manually. Our technical evaluation shows that our custom pipelines produce more accurate outdoor scene graphs and generate 3D meshes faster than prior methods. A three-part user study (N=20) revealed preferred roles for AI, how users create in freeform use, and design implications for future AR authoring tools. ImaginateAR takes a step toward empowering anyone to create AR experiences anywhere -- simply by speaking their imagination. |
| title | ImaginateAR: AI-Assisted In-Situ Authoring in Augmented Reality |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2504.21360 |