ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908338521899008 |
|---|---|
| author | Xia, Renqiu Zhang, Bo Ye, Hancheng Yan, Xiangchao Liu, Qi Zhou, Hongbin Chen, Zijun Ye, Peng Dou, Min Shi, Botian Yan, Junchi Qiao, Yu |
| author_facet | Xia, Renqiu Zhang, Bo Ye, Hancheng Yan, Xiangchao Liu, Qi Zhou, Hongbin Chen, Zijun Ye, Peng Dou, Min Shi, Botian Yan, Junchi Qiao, Yu |
| contents | Recently, many versatile Multi-modal Large Language Models (MLLMs) have emerged continuously. However, their capacity to query information depicted in visual charts and engage in reasoning based on the queried contents remains under-explored. In this paper, to comprehensively and rigorously benchmark the ability of the off-the-shelf MLLMs in the chart domain, we construct ChartX, a multi-modal evaluation set covering 18 chart types, 7 chart tasks, 22 disciplinary topics, and high-quality chart data. Besides, we develop ChartVLM to offer a new perspective on handling multi-modal tasks that strongly depend on interpretable patterns, such as reasoning tasks in the field of charts or geometric images. We evaluate the chart-related ability of mainstream MLLMs and our ChartVLM on the proposed ChartX evaluation set. Extensive experiments demonstrate that ChartVLM surpasses both versatile and chart-related large models, achieving results comparable to GPT-4V. We believe that our study can pave the way for further exploration in creating a more comprehensive chart evaluation set and developing more interpretable multi-modal models. Both ChartX and ChartVLM are available at: https://github.com/Alpha-Innovator/ChartVLM |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_12185 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning Xia, Renqiu Zhang, Bo Ye, Hancheng Yan, Xiangchao Liu, Qi Zhou, Hongbin Chen, Zijun Ye, Peng Dou, Min Shi, Botian Yan, Junchi Qiao, Yu Computer Vision and Pattern Recognition Recently, many versatile Multi-modal Large Language Models (MLLMs) have emerged continuously. However, their capacity to query information depicted in visual charts and engage in reasoning based on the queried contents remains under-explored. In this paper, to comprehensively and rigorously benchmark the ability of the off-the-shelf MLLMs in the chart domain, we construct ChartX, a multi-modal evaluation set covering 18 chart types, 7 chart tasks, 22 disciplinary topics, and high-quality chart data. Besides, we develop ChartVLM to offer a new perspective on handling multi-modal tasks that strongly depend on interpretable patterns, such as reasoning tasks in the field of charts or geometric images. We evaluate the chart-related ability of mainstream MLLMs and our ChartVLM on the proposed ChartX evaluation set. Extensive experiments demonstrate that ChartVLM surpasses both versatile and chart-related large models, achieving results comparable to GPT-4V. We believe that our study can pave the way for further exploration in creating a more comprehensive chart evaluation set and developing more interpretable multi-modal models. Both ChartX and ChartVLM are available at: https://github.com/Alpha-Innovator/ChartVLM |
| title | ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.12185 |