VIS-Shepherd: Constructing Critic for LLM-based Data Visualization Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916795631271936 |
|---|---|
| author | Pan, Bo Fu, Yixiao Wang, Ke Lu, Junyu Pan, Lunke Qian, Ziyang Chen, Yuhan Wang, Guoliang Zhou, Yitao Zheng, Li Tang, Yinghao Wen, Zhen Wu, Yuchen Lu, Junhua Zhu, Biao Zhu, Minfeng Zhang, Bo Chen, Wei |
| author_facet | Pan, Bo Fu, Yixiao Wang, Ke Lu, Junyu Pan, Lunke Qian, Ziyang Chen, Yuhan Wang, Guoliang Zhou, Yitao Zheng, Li Tang, Yinghao Wen, Zhen Wu, Yuchen Lu, Junhua Zhu, Biao Zhu, Minfeng Zhang, Bo Chen, Wei |
| contents | Data visualization generation using Large Language Models (LLMs) has shown promising results but often produces suboptimal visualizations that require human intervention for improvement. In this work, we introduce VIS-Shepherd, a specialized Multimodal Large Language Model (MLLM)-based critic to evaluate and provide feedback for LLM-generated data visualizations. At the core of our approach is a framework to construct a high-quality visualization critique dataset, where we collect human-created visualization instances, synthesize corresponding LLM-generated instances, and construct high-quality critiques. We conduct both model-based automatic evaluation and human preference studies to evaluate the effectiveness of our approach. Our experiments show that even small (7B parameters) open-source MLLM models achieve substantial performance gains by leveraging our high-quality visualization critique dataset, reaching levels comparable to much larger open-source or even proprietary models. Our work demonstrates significant potential for MLLM-based automated visualization critique and indicates promising directions for enhancing LLM-based data visualization generation. Our project page: https://github.com/bopan3/VIS-Shepherd. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_13326 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VIS-Shepherd: Constructing Critic for LLM-based Data Visualization Generation Pan, Bo Fu, Yixiao Wang, Ke Lu, Junyu Pan, Lunke Qian, Ziyang Chen, Yuhan Wang, Guoliang Zhou, Yitao Zheng, Li Tang, Yinghao Wen, Zhen Wu, Yuchen Lu, Junhua Zhu, Biao Zhu, Minfeng Zhang, Bo Chen, Wei Computer Vision and Pattern Recognition Human-Computer Interaction Data visualization generation using Large Language Models (LLMs) has shown promising results but often produces suboptimal visualizations that require human intervention for improvement. In this work, we introduce VIS-Shepherd, a specialized Multimodal Large Language Model (MLLM)-based critic to evaluate and provide feedback for LLM-generated data visualizations. At the core of our approach is a framework to construct a high-quality visualization critique dataset, where we collect human-created visualization instances, synthesize corresponding LLM-generated instances, and construct high-quality critiques. We conduct both model-based automatic evaluation and human preference studies to evaluate the effectiveness of our approach. Our experiments show that even small (7B parameters) open-source MLLM models achieve substantial performance gains by leveraging our high-quality visualization critique dataset, reaching levels comparable to much larger open-source or even proprietary models. Our work demonstrates significant potential for MLLM-based automated visualization critique and indicates promising directions for enhancing LLM-based data visualization generation. Our project page: https://github.com/bopan3/VIS-Shepherd. |
| title | VIS-Shepherd: Constructing Critic for LLM-based Data Visualization Generation |
| topic | Computer Vision and Pattern Recognition Human-Computer Interaction |
| url | https://arxiv.org/abs/2506.13326 |