StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesis
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911767856152576 |
|---|---|
| author | Tang, Zecheng Wu, Chenfei Zhang, Zekai Ni, Mingheng Yin, Shengming Liu, Yu Yang, Zhengyuan Wang, Lijuan Liu, Zicheng Li, Juntao Duan, Nan |
| author_facet | Tang, Zecheng Wu, Chenfei Zhang, Zekai Ni, Mingheng Yin, Shengming Liu, Yu Yang, Zhengyuan Wang, Lijuan Liu, Zicheng Li, Juntao Duan, Nan |
| contents | To leverage LLMs for visual synthesis, traditional methods convert raster image information into discrete grid tokens through specialized visual modules, while disrupting the model's ability to capture the true semantic representation of visual scenes. This paper posits that an alternative representation of images, vector graphics, can effectively surmount this limitation by enabling a more natural and semantically coherent segmentation of the image information. Thus, we introduce StrokeNUWA, a pioneering work exploring a better visual representation ''stroke tokens'' on vector graphics, which is inherently visual semantics rich, naturally compatible with LLMs, and highly compressed. Equipped with stroke tokens, StrokeNUWA can significantly surpass traditional LLM-based and optimization-based methods across various metrics in the vector graphic generation task. Besides, StrokeNUWA achieves up to a 94x speedup in inference over the speed of prior methods with an exceptional SVG code compression ratio of 6.9%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_17093 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesis Tang, Zecheng Wu, Chenfei Zhang, Zekai Ni, Mingheng Yin, Shengming Liu, Yu Yang, Zhengyuan Wang, Lijuan Liu, Zicheng Li, Juntao Duan, Nan Computer Vision and Pattern Recognition Computation and Language To leverage LLMs for visual synthesis, traditional methods convert raster image information into discrete grid tokens through specialized visual modules, while disrupting the model's ability to capture the true semantic representation of visual scenes. This paper posits that an alternative representation of images, vector graphics, can effectively surmount this limitation by enabling a more natural and semantically coherent segmentation of the image information. Thus, we introduce StrokeNUWA, a pioneering work exploring a better visual representation ''stroke tokens'' on vector graphics, which is inherently visual semantics rich, naturally compatible with LLMs, and highly compressed. Equipped with stroke tokens, StrokeNUWA can significantly surpass traditional LLM-based and optimization-based methods across various metrics in the vector graphic generation task. Besides, StrokeNUWA achieves up to a 94x speedup in inference over the speed of prior methods with an exceptional SVG code compression ratio of 6.9%. |
| title | StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesis |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2401.17093 |