How Far Can Off-the-Shelf Multimodal Large Language Models Go in Online Episodic Memory Question Answering?
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912441913311232 |
|---|---|
| author | Lando, Giuseppe Forte, Rosario Farinella, Giovanni Maria Furnari, Antonino |
| author_facet | Lando, Giuseppe Forte, Rosario Farinella, Giovanni Maria Furnari, Antonino |
| contents | We investigate whether off-the-shelf Multimodal Large Language Models (MLLMs) can tackle Online Episodic-Memory Video Question Answering (OEM-VQA) without additional training. Our pipeline converts a streaming egocentric video into a lightweight textual memory, only a few kilobytes per minute, via an MLLM descriptor module, and answers multiple-choice questions by querying this memory with an LLM reasoner module. On the QAEgo4D-Closed benchmark, our best configuration attains 56.0% accuracy with 3.6 kB per minute storage, matching the performance of dedicated state-of-the-art systems while being 10**4/10**5 times more memory-efficient. Extensive ablations provides insights into the role of each component and design choice, and highlight directions of improvement for future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_16450 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | How Far Can Off-the-Shelf Multimodal Large Language Models Go in Online Episodic Memory Question Answering? Lando, Giuseppe Forte, Rosario Farinella, Giovanni Maria Furnari, Antonino Computer Vision and Pattern Recognition We investigate whether off-the-shelf Multimodal Large Language Models (MLLMs) can tackle Online Episodic-Memory Video Question Answering (OEM-VQA) without additional training. Our pipeline converts a streaming egocentric video into a lightweight textual memory, only a few kilobytes per minute, via an MLLM descriptor module, and answers multiple-choice questions by querying this memory with an LLM reasoner module. On the QAEgo4D-Closed benchmark, our best configuration attains 56.0% accuracy with 3.6 kB per minute storage, matching the performance of dedicated state-of-the-art systems while being 10**4/10**5 times more memory-efficient. Extensive ablations provides insights into the role of each component and design choice, and highlight directions of improvement for future research. |
| title | How Far Can Off-the-Shelf Multimodal Large Language Models Go in Online Episodic Memory Question Answering? |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2506.16450 |