StructChart: On the Schema, Metric, and Augmentation for Visual Chart Understanding
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910725923930112 |
|---|---|
| author | Xia, Renqiu Peng, Haoyang Ye, Hancheng Li, Mingsheng Yan, Xiangchao Ye, Peng Shi, Botian Qiao, Yu Yan, Junchi Zhang, Bo |
| author_facet | Xia, Renqiu Peng, Haoyang Ye, Hancheng Li, Mingsheng Yan, Xiangchao Ye, Peng Shi, Botian Qiao, Yu Yan, Junchi Zhang, Bo |
| contents | Charts are common in literature across various scientific fields, conveying rich information easily accessible to readers. Current chart-related tasks focus on either chart perception that extracts information from the visual charts, or chart reasoning given the extracted data, e.g. in a tabular form. In this paper, we introduce StructChart, a novel framework that leverages Structured Triplet Representations (STR) to achieve a unified and label-efficient approach to chart perception and reasoning tasks, which is generally applicable to different downstream tasks, beyond the question-answering task as specifically studied in peer works. Specifically, StructChart first reformulates the chart data from the tubular form (linearized CSV) to STR, which can friendlily reduce the task gap between chart perception and reasoning. We then propose a Structuring Chart-oriented Representation Metric (SCRM) to quantitatively evaluate the chart perception task performance. To augment the training, we further explore the potential of Large Language Models (LLMs) to enhance the diversity in both chart visual style and statistical information. Extensive experiments on various chart-related tasks demonstrate the effectiveness and potential of a unified chart perception-reasoning paradigm to push the frontier of chart understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_11268 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | StructChart: On the Schema, Metric, and Augmentation for Visual Chart Understanding Xia, Renqiu Peng, Haoyang Ye, Hancheng Li, Mingsheng Yan, Xiangchao Ye, Peng Shi, Botian Qiao, Yu Yan, Junchi Zhang, Bo Computer Vision and Pattern Recognition Charts are common in literature across various scientific fields, conveying rich information easily accessible to readers. Current chart-related tasks focus on either chart perception that extracts information from the visual charts, or chart reasoning given the extracted data, e.g. in a tabular form. In this paper, we introduce StructChart, a novel framework that leverages Structured Triplet Representations (STR) to achieve a unified and label-efficient approach to chart perception and reasoning tasks, which is generally applicable to different downstream tasks, beyond the question-answering task as specifically studied in peer works. Specifically, StructChart first reformulates the chart data from the tubular form (linearized CSV) to STR, which can friendlily reduce the task gap between chart perception and reasoning. We then propose a Structuring Chart-oriented Representation Metric (SCRM) to quantitatively evaluate the chart perception task performance. To augment the training, we further explore the potential of Large Language Models (LLMs) to enhance the diversity in both chart visual style and statistical information. Extensive experiments on various chart-related tasks demonstrate the effectiveness and potential of a unified chart perception-reasoning paradigm to push the frontier of chart understanding. |
| title | StructChart: On the Schema, Metric, and Augmentation for Visual Chart Understanding |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2309.11268 |