Beyond Generating Code: Evaluating GPT on a Data Visualization Course
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910443414487040 |
|---|---|
| author | Zhu-Tian, Chen Zhang, Chenyang Wang, Qianwen Troidl, Jakob Warchol, Simon Beyer, Johanna Gehlenborg, Nils Pfister, Hanspeter |
| author_facet | Zhu-Tian, Chen Zhang, Chenyang Wang, Qianwen Troidl, Jakob Warchol, Simon Beyer, Johanna Gehlenborg, Nils Pfister, Hanspeter |
| contents | This paper presents an empirical evaluation of the performance of the Generative Pre-trained Transformer (GPT) model in Harvard's CS171 data visualization course. While previous studies have focused on GPT's ability to generate code for visualizations, this study goes beyond code generation to evaluate GPT's abilities in various visualization tasks, such as data interpretation, visualization design, visual data exploration, and insight communication. The evaluation utilized GPT-3.5 and GPT-4 to complete assignments of CS171, and included a quantitative assessment based on the established course rubrics, a qualitative analysis informed by the feedback of three experienced graders, and an exploratory study of GPT's capabilities in completing border visualization tasks. Findings show that GPT-4 scored 80% on quizzes and homework, and TFs could distinguish between GPT- and human-generated homework with 70% accuracy. The study also demonstrates GPT's potential in completing various visualization tasks, such as data cleanup, interaction with visualizations, and insight communication. The paper concludes by discussing the strengths and limitations of GPT in data visualization, potential avenues for incorporating GPT in broader visualization tasks, and the need to redesign visualization education. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_02914 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Beyond Generating Code: Evaluating GPT on a Data Visualization Course Zhu-Tian, Chen Zhang, Chenyang Wang, Qianwen Troidl, Jakob Warchol, Simon Beyer, Johanna Gehlenborg, Nils Pfister, Hanspeter Human-Computer Interaction Graphics This paper presents an empirical evaluation of the performance of the Generative Pre-trained Transformer (GPT) model in Harvard's CS171 data visualization course. While previous studies have focused on GPT's ability to generate code for visualizations, this study goes beyond code generation to evaluate GPT's abilities in various visualization tasks, such as data interpretation, visualization design, visual data exploration, and insight communication. The evaluation utilized GPT-3.5 and GPT-4 to complete assignments of CS171, and included a quantitative assessment based on the established course rubrics, a qualitative analysis informed by the feedback of three experienced graders, and an exploratory study of GPT's capabilities in completing border visualization tasks. Findings show that GPT-4 scored 80% on quizzes and homework, and TFs could distinguish between GPT- and human-generated homework with 70% accuracy. The study also demonstrates GPT's potential in completing various visualization tasks, such as data cleanup, interaction with visualizations, and insight communication. The paper concludes by discussing the strengths and limitations of GPT in data visualization, potential avenues for incorporating GPT in broader visualization tasks, and the need to redesign visualization education. |
| title | Beyond Generating Code: Evaluating GPT on a Data Visualization Course |
| topic | Human-Computer Interaction Graphics |
| url | https://arxiv.org/abs/2306.02914 |