AR Secretary Agent: Real-time Memory Augmentation via LLM-powered Augmented Reality Glasses
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908368934797312 |
|---|---|
| author | Haddad, Raphaël A. El Wang, Zeyu Shin, Yeonsu Liu, Ranyi Wang, Yuntao Yu, Chun |
| author_facet | Haddad, Raphaël A. El Wang, Zeyu Shin, Yeonsu Liu, Ranyi Wang, Yuntao Yu, Chun |
| contents | Interacting with a significant number of individuals on a daily basis is commonplace for many professionals, which can lead to challenges in recalling specific details: Who is this person? What did we talk about last time? The advant of augmented reality (AR) glasses, equipped with visual and auditory data capture capabilities, presents a solution. In our work, we implemented an AR Secretary Agent with advanced Large Language Models (LLMs) and Computer Vision technologies. This system could discreetly provide real-time information to the wearer, identifying who they are conversing with and summarizing previous discussions. To verify AR Secretary, we conducted a user study with 13 participants and showed that our technique can efficiently help users to memorize events by up to 20\% memory enhancement on our study. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_11888 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AR Secretary Agent: Real-time Memory Augmentation via LLM-powered Augmented Reality Glasses Haddad, Raphaël A. El Wang, Zeyu Shin, Yeonsu Liu, Ranyi Wang, Yuntao Yu, Chun Human-Computer Interaction Interacting with a significant number of individuals on a daily basis is commonplace for many professionals, which can lead to challenges in recalling specific details: Who is this person? What did we talk about last time? The advant of augmented reality (AR) glasses, equipped with visual and auditory data capture capabilities, presents a solution. In our work, we implemented an AR Secretary Agent with advanced Large Language Models (LLMs) and Computer Vision technologies. This system could discreetly provide real-time information to the wearer, identifying who they are conversing with and summarizing previous discussions. To verify AR Secretary, we conducted a user study with 13 participants and showed that our technique can efficiently help users to memorize events by up to 20\% memory enhancement on our study. |
| title | AR Secretary Agent: Real-time Memory Augmentation via LLM-powered Augmented Reality Glasses |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2505.11888 |