From Words to Structured Visuals: A Benchmark and Framework for Text-to-Diagram Generation and Editing
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912123865530368 |
|---|---|
| author | Wei, Jingxuan Tan, Cheng Chen, Qi Wu, Gaowei Li, Siyuan Gao, Zhangyang Sun, Linzhuang Yu, Bihui Guo, Ruifeng |
| author_facet | Wei, Jingxuan Tan, Cheng Chen, Qi Wu, Gaowei Li, Siyuan Gao, Zhangyang Sun, Linzhuang Yu, Bihui Guo, Ruifeng |
| contents | We introduce the task of text-to-diagram generation, which focuses on creating structured visual representations directly from textual descriptions. Existing approaches in text-to-image and text-to-code generation lack the logical organization and flexibility needed to produce accurate, editable diagrams, often resulting in outputs that are either unstructured or difficult to modify. To address this gap, we introduce DiagramGenBenchmark, a comprehensive evaluation framework encompassing eight distinct diagram categories, including flowcharts, model architecture diagrams, and mind maps. Additionally, we present DiagramAgent, an innovative framework with four core modules-Plan Agent, Code Agent, Check Agent, and Diagram-to-Code Agent-designed to facilitate both the generation and refinement of complex diagrams. Our extensive experiments, which combine objective metrics with human evaluations, demonstrate that DiagramAgent significantly outperforms existing baseline models in terms of accuracy, structural coherence, and modifiability. This work not only establishes a foundational benchmark for the text-to-diagram generation task but also introduces a powerful toolset to advance research and applications in this emerging area. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_11916 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | From Words to Structured Visuals: A Benchmark and Framework for Text-to-Diagram Generation and Editing Wei, Jingxuan Tan, Cheng Chen, Qi Wu, Gaowei Li, Siyuan Gao, Zhangyang Sun, Linzhuang Yu, Bihui Guo, Ruifeng Databases We introduce the task of text-to-diagram generation, which focuses on creating structured visual representations directly from textual descriptions. Existing approaches in text-to-image and text-to-code generation lack the logical organization and flexibility needed to produce accurate, editable diagrams, often resulting in outputs that are either unstructured or difficult to modify. To address this gap, we introduce DiagramGenBenchmark, a comprehensive evaluation framework encompassing eight distinct diagram categories, including flowcharts, model architecture diagrams, and mind maps. Additionally, we present DiagramAgent, an innovative framework with four core modules-Plan Agent, Code Agent, Check Agent, and Diagram-to-Code Agent-designed to facilitate both the generation and refinement of complex diagrams. Our extensive experiments, which combine objective metrics with human evaluations, demonstrate that DiagramAgent significantly outperforms existing baseline models in terms of accuracy, structural coherence, and modifiability. This work not only establishes a foundational benchmark for the text-to-diagram generation task but also introduces a powerful toolset to advance research and applications in this emerging area. |
| title | From Words to Structured Visuals: A Benchmark and Framework for Text-to-Diagram Generation and Editing |
| topic | Databases |
| url | https://arxiv.org/abs/2411.11916 |