ChartLens: Fine-grained Visual Attribution in Charts
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915304293007360 |
|---|---|
| author | Suri, Manan Mathur, Puneet Lipka, Nedim Dernoncourt, Franck Rossi, Ryan A. Manocha, Dinesh |
| author_facet | Suri, Manan Mathur, Puneet Lipka, Nedim Dernoncourt, Franck Rossi, Ryan A. Manocha, Dinesh |
| contents | The growing capabilities of multimodal large language models (MLLMs) have advanced tasks like chart understanding. However, these models often suffer from hallucinations, where generated text sequences conflict with the provided visual data. To address this, we introduce Post-Hoc Visual Attribution for Charts, which identifies fine-grained chart elements that validate a given chart-associated response. We propose ChartLens, a novel chart attribution algorithm that uses segmentation-based techniques to identify chart objects and employs set-of-marks prompting with MLLMs for fine-grained visual attribution. Additionally, we present ChartVA-Eval, a benchmark with synthetic and real-world charts from diverse domains like finance, policy, and economics, featuring fine-grained attribution annotations. Our evaluations show that ChartLens improves fine-grained attributions by 26-66%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_19360 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ChartLens: Fine-grained Visual Attribution in Charts Suri, Manan Mathur, Puneet Lipka, Nedim Dernoncourt, Franck Rossi, Ryan A. Manocha, Dinesh Computation and Language The growing capabilities of multimodal large language models (MLLMs) have advanced tasks like chart understanding. However, these models often suffer from hallucinations, where generated text sequences conflict with the provided visual data. To address this, we introduce Post-Hoc Visual Attribution for Charts, which identifies fine-grained chart elements that validate a given chart-associated response. We propose ChartLens, a novel chart attribution algorithm that uses segmentation-based techniques to identify chart objects and employs set-of-marks prompting with MLLMs for fine-grained visual attribution. Additionally, we present ChartVA-Eval, a benchmark with synthetic and real-world charts from diverse domains like finance, policy, and economics, featuring fine-grained attribution annotations. Our evaluations show that ChartLens improves fine-grained attributions by 26-66%. |
| title | ChartLens: Fine-grained Visual Attribution in Charts |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.19360 |