From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909043955597312 |
|---|---|
| author | Chen, Guanhua Huang, Chuyue Yao, Yutong Liu, Shudong Song, Xueqing Chao, Lidia S. Wong, Derek F. |
| author_facet | Chen, Guanhua Huang, Chuyue Yao, Yutong Liu, Shudong Song, Xueqing Chao, Lidia S. Wong, Derek F. |
| contents | Multimodal Retrieval-Augmented Generation (RAG) systems retrieve evidence at coarse granularities (entire images or scenes), creating a mismatch with fine-grained user queries and making failures unverifiable. We introduce GranuVistaVQA, a multimodal benchmark featuring real-world landmarks with element-level annotations across multiple viewpoints, capturing the partial observation challenge where individual images contain only subsets of entities. We further propose GranuRAG, a multi-granularity framework that treats visual elements as first-class retrieval units through three stages: element-level detection and classification, multi-granularity cross-modal alignment for evidence retrieval, and attribution-constrained generation. By grounding retrieval at the element level rather than relying on implicit attention, our approach enables transparent error diagnosis. Experiments demonstrate that GranuRAG achieves up to 29.2% improvement over six strong baselines for this task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_15019 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG Chen, Guanhua Huang, Chuyue Yao, Yutong Liu, Shudong Song, Xueqing Chao, Lidia S. Wong, Derek F. Computation and Language Multimodal Retrieval-Augmented Generation (RAG) systems retrieve evidence at coarse granularities (entire images or scenes), creating a mismatch with fine-grained user queries and making failures unverifiable. We introduce GranuVistaVQA, a multimodal benchmark featuring real-world landmarks with element-level annotations across multiple viewpoints, capturing the partial observation challenge where individual images contain only subsets of entities. We further propose GranuRAG, a multi-granularity framework that treats visual elements as first-class retrieval units through three stages: element-level detection and classification, multi-granularity cross-modal alignment for evidence retrieval, and attribution-constrained generation. By grounding retrieval at the element level rather than relying on implicit attention, our approach enables transparent error diagnosis. Experiments demonstrate that GranuRAG achieves up to 29.2% improvement over six strong baselines for this task. |
| title | From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2605.15019 |