DiagramIR: An Automatic Pipeline for Educational Math Diagram Evaluation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915611990294528 |
|---|---|
| author | Kumar, Vishal Mishra, Shubhra Hao, Rebecca Malik, Rizwaan Broman, David Demszky, Dorottya |
| author_facet | Kumar, Vishal Mishra, Shubhra Hao, Rebecca Malik, Rizwaan Broman, David Demszky, Dorottya |
| contents | Large Language Models (LLMs) are increasingly being adopted as tools for learning; however, most tools remain text-only, limiting their usefulness for domains where visualizations are essential, such as mathematics. Recent work shows that LLMs are capable of generating code that compiles to educational figures, but a major bottleneck remains: scalable evaluation of these diagrams. We address this by proposing DiagramIR: an automatic and scalable evaluation pipeline for geometric figures. Our method relies on intermediate representations (IRs) of LaTeX TikZ code. We compare our pipeline to other evaluation baselines such as LLM-as-a-Judge, showing that our approach has higher agreement with human raters. This evaluation approach also enables smaller models like GPT-4.1-Mini to perform comparably to larger models such as GPT-5 at a 10x lower inference cost, which is important for deploying accessible and scalable education technologies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_08283 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DiagramIR: An Automatic Pipeline for Educational Math Diagram Evaluation Kumar, Vishal Mishra, Shubhra Hao, Rebecca Malik, Rizwaan Broman, David Demszky, Dorottya Artificial Intelligence Large Language Models (LLMs) are increasingly being adopted as tools for learning; however, most tools remain text-only, limiting their usefulness for domains where visualizations are essential, such as mathematics. Recent work shows that LLMs are capable of generating code that compiles to educational figures, but a major bottleneck remains: scalable evaluation of these diagrams. We address this by proposing DiagramIR: an automatic and scalable evaluation pipeline for geometric figures. Our method relies on intermediate representations (IRs) of LaTeX TikZ code. We compare our pipeline to other evaluation baselines such as LLM-as-a-Judge, showing that our approach has higher agreement with human raters. This evaluation approach also enables smaller models like GPT-4.1-Mini to perform comparably to larger models such as GPT-5 at a 10x lower inference cost, which is important for deploying accessible and scalable education technologies. |
| title | DiagramIR: An Automatic Pipeline for Educational Math Diagram Evaluation |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2511.08283 |