Spatially Grounded Explanations in Vision Language Models for Document Visual Question Answering
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912487787462656 |
|---|---|
| author | Lagos, Maximiliano Hormazábal Cerezo-Costas, Héctor Karatzas, Dimosthenis |
| author_facet | Lagos, Maximiliano Hormazábal Cerezo-Costas, Héctor Karatzas, Dimosthenis |
| contents | We introduce EaGERS, a fully training-free and model-agnostic pipeline that (1) generates natural language rationales via a vision language model, (2) grounds these rationales to spatial sub-regions by computing multimodal embedding similarities over a configurable grid with majority voting, and (3) restricts the generation of responses only from the relevant regions selected in the masked image. Experiments on the DocVQA dataset demonstrate that our best configuration not only outperforms the base model on exact match accuracy and Average Normalized Levenshtein Similarity metrics but also enhances transparency and reproducibility in DocVQA without additional model fine-tuning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_12490 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Spatially Grounded Explanations in Vision Language Models for Document Visual Question Answering Lagos, Maximiliano Hormazábal Cerezo-Costas, Héctor Karatzas, Dimosthenis Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Machine Learning We introduce EaGERS, a fully training-free and model-agnostic pipeline that (1) generates natural language rationales via a vision language model, (2) grounds these rationales to spatial sub-regions by computing multimodal embedding similarities over a configurable grid with majority voting, and (3) restricts the generation of responses only from the relevant regions selected in the masked image. Experiments on the DocVQA dataset demonstrate that our best configuration not only outperforms the base model on exact match accuracy and Average Normalized Levenshtein Similarity metrics but also enhances transparency and reproducibility in DocVQA without additional model fine-tuning. |
| title | Spatially Grounded Explanations in Vision Language Models for Document Visual Question Answering |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2507.12490 |