AdjustAR: AI-Driven In-Situ Adjustment of Site-Specific Augmented Reality Content
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908483590291456 |
|---|---|
| author | Numan, Nels Van Brummelen, Jessica Lu, Ziwen Steed, Anthony |
| author_facet | Numan, Nels Van Brummelen, Jessica Lu, Ziwen Steed, Anthony |
| contents | Site-specific outdoor AR experiences are typically authored using static 3D models, but are deployed in physical environments that change over time. As a result, virtual content may become misaligned with its intended real-world referents, degrading user experience and compromising contextual interpretation. We present AdjustAR, a system that supports in-situ correction of AR content in dynamic environments using multimodal large language models (MLLMs). Given a composite image comprising the originally authored view and the current live user view from the same perspective, an MLLM detects contextual misalignments and proposes revised 2D placements for affected AR elements. These corrections are backprojected into 3D space to update the scene at runtime. By leveraging MLLMs for visual-semantic reasoning, this approach enables automated runtime corrections to maintain alignment with the authored intent as real-world target environments evolve. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_06826 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AdjustAR: AI-Driven In-Situ Adjustment of Site-Specific Augmented Reality Content Numan, Nels Van Brummelen, Jessica Lu, Ziwen Steed, Anthony Human-Computer Interaction Site-specific outdoor AR experiences are typically authored using static 3D models, but are deployed in physical environments that change over time. As a result, virtual content may become misaligned with its intended real-world referents, degrading user experience and compromising contextual interpretation. We present AdjustAR, a system that supports in-situ correction of AR content in dynamic environments using multimodal large language models (MLLMs). Given a composite image comprising the originally authored view and the current live user view from the same perspective, an MLLM detects contextual misalignments and proposes revised 2D placements for affected AR elements. These corrections are backprojected into 3D space to update the scene at runtime. By leveraging MLLMs for visual-semantic reasoning, this approach enables automated runtime corrections to maintain alignment with the authored intent as real-world target environments evolve. |
| title | AdjustAR: AI-Driven In-Situ Adjustment of Site-Specific Augmented Reality Content |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2508.06826 |