Enhancing Multi-Image Question Answering via Submodular Subset Selection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910946459385856 |
|---|---|
| author | Sharma, Aaryan Gupta, Shivansh Agarwal, Samar C., Vishak Prasad Ramakrishnan, Ganesh |
| author_facet | Sharma, Aaryan Gupta, Shivansh Agarwal, Samar C., Vishak Prasad Ramakrishnan, Ganesh |
| contents | Large multimodal models (LMMs) have achieved high performance in vision-language tasks involving single image but they struggle when presented with a collection of multiple images (Multiple Image Question Answering scenario). These tasks, which involve reasoning over large number of images, present issues in scalability (with increasing number of images) and retrieval performance. In this work, we propose an enhancement for retriever framework introduced in MIRAGE model using submodular subset selection techniques. Our method leverages query-aware submodular functions, such as GraphCut, to pre-select a subset of semantically relevant images before main retrieval component. We demonstrate that using anchor-based queries and augmenting the data improves submodular-retriever pipeline effectiveness, particularly in large haystack sizes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_10533 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Enhancing Multi-Image Question Answering via Submodular Subset Selection Sharma, Aaryan Gupta, Shivansh Agarwal, Samar C., Vishak Prasad Ramakrishnan, Ganesh Computer Vision and Pattern Recognition Machine Learning Large multimodal models (LMMs) have achieved high performance in vision-language tasks involving single image but they struggle when presented with a collection of multiple images (Multiple Image Question Answering scenario). These tasks, which involve reasoning over large number of images, present issues in scalability (with increasing number of images) and retrieval performance. In this work, we propose an enhancement for retriever framework introduced in MIRAGE model using submodular subset selection techniques. Our method leverages query-aware submodular functions, such as GraphCut, to pre-select a subset of semantically relevant images before main retrieval component. We demonstrate that using anchor-based queries and augmenting the data improves submodular-retriever pipeline effectiveness, particularly in large haystack sizes. |
| title | Enhancing Multi-Image Question Answering via Submodular Subset Selection |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2505.10533 |