Insight: Enhancing Mobile Accessibility for Blind and Visually Impaired Users with LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914551668146176 |
|---|---|
| author | Ansah, Joshua Owusu Kapoor, Anuj Khanna, Ayush Vinod, Manvika Njeck, Precious Gao, Shuai |
| author_facet | Ansah, Joshua Owusu Kapoor, Anuj Khanna, Ayush Vinod, Manvika Njeck, Precious Gao, Shuai |
| contents | This research paper addresses the limitations of current mobile accessibility services like TalkBack, which provide manual gesture-based sequential feedback to BVI users. Motivated by the promise of large language models (LLMs), this paper introduces Insight, an Android accessibility service that provides natural language interaction and real-time summarization of the screen. The paper performs a within-subject experimental study with users to compare Insight and TalkBack on usability factors. Results show Insight reduced mental effort and task time, and was preferred because of its dialogue interface, but users felt the need for interruption management. Results show LLM-based interfaces can significantly improve mobile accessibility, and describe the potential of hybrid solutions combining gesture and dialogue modalities towards more inclusive design. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_09803 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Insight: Enhancing Mobile Accessibility for Blind and Visually Impaired Users with LLMs Ansah, Joshua Owusu Kapoor, Anuj Khanna, Ayush Vinod, Manvika Njeck, Precious Gao, Shuai Human-Computer Interaction Artificial Intelligence This research paper addresses the limitations of current mobile accessibility services like TalkBack, which provide manual gesture-based sequential feedback to BVI users. Motivated by the promise of large language models (LLMs), this paper introduces Insight, an Android accessibility service that provides natural language interaction and real-time summarization of the screen. The paper performs a within-subject experimental study with users to compare Insight and TalkBack on usability factors. Results show Insight reduced mental effort and task time, and was preferred because of its dialogue interface, but users felt the need for interruption management. Results show LLM-based interfaces can significantly improve mobile accessibility, and describe the potential of hybrid solutions combining gesture and dialogue modalities towards more inclusive design. |
| title | Insight: Enhancing Mobile Accessibility for Blind and Visually Impaired Users with LLMs |
| topic | Human-Computer Interaction Artificial Intelligence |
| url | https://arxiv.org/abs/2605.09803 |