EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answering
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917964340527104 |
|---|---|
| author | Zhou, Sheng Xiao, Junbin Li, Qingyun Li, Yicong Yang, Xun Guo, Dan Wang, Meng Chua, Tat-Seng Yao, Angela |
| author_facet | Zhou, Sheng Xiao, Junbin Li, Qingyun Li, Yicong Yang, Xun Guo, Dan Wang, Meng Chua, Tat-Seng Yao, Angela |
| contents | We introduce EgoTextVQA, a novel and rigorously constructed benchmark for egocentric QA assistance involving scene text. EgoTextVQA contains 1.5K ego-view videos and 7K scene-text aware questions that reflect real user needs in outdoor driving and indoor house-keeping activities. The questions are designed to elicit identification and reasoning on scene text in an egocentric and dynamic environment. With EgoTextVQA, we comprehensively evaluate 10 prominent multimodal large language models. Currently, all models struggle, and the best results (Gemini 1.5 Pro) are around 33\% accuracy, highlighting the severe deficiency of these techniques in egocentric QA assistance. Our further investigations suggest that precise temporal grounding and multi-frame reasoning, along with high resolution and auxiliary scene-text inputs, are key for better performance. With thorough analyses and heuristic suggestions, we hope EgoTextVQA can serve as a solid testbed for research in egocentric scene-text QA assistance. Our dataset is released at: https://github.com/zhousheng97/EgoTextVQA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_07411 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answering Zhou, Sheng Xiao, Junbin Li, Qingyun Li, Yicong Yang, Xun Guo, Dan Wang, Meng Chua, Tat-Seng Yao, Angela Computer Vision and Pattern Recognition Multimedia We introduce EgoTextVQA, a novel and rigorously constructed benchmark for egocentric QA assistance involving scene text. EgoTextVQA contains 1.5K ego-view videos and 7K scene-text aware questions that reflect real user needs in outdoor driving and indoor house-keeping activities. The questions are designed to elicit identification and reasoning on scene text in an egocentric and dynamic environment. With EgoTextVQA, we comprehensively evaluate 10 prominent multimodal large language models. Currently, all models struggle, and the best results (Gemini 1.5 Pro) are around 33\% accuracy, highlighting the severe deficiency of these techniques in egocentric QA assistance. Our further investigations suggest that precise temporal grounding and multi-frame reasoning, along with high resolution and auxiliary scene-text inputs, are key for better performance. With thorough analyses and heuristic suggestions, we hope EgoTextVQA can serve as a solid testbed for research in egocentric scene-text QA assistance. Our dataset is released at: https://github.com/zhousheng97/EgoTextVQA. |
| title | EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answering |
| topic | Computer Vision and Pattern Recognition Multimedia |
| url | https://arxiv.org/abs/2502.07411 |