CLIPDraw++: Text-to-Sketch Synthesis with Simple Primitives
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909679899115520 |
|---|---|
| author | Mathur, Nityanand Marjit, Shyam Chaudhuri, Abhra Dutta, Anjan |
| author_facet | Mathur, Nityanand Marjit, Shyam Chaudhuri, Abhra Dutta, Anjan |
| contents | With the goal of understanding the visual concepts that CLIP associates with text prompts, we show that the latent space of CLIP can be visualized solely in terms of linear transformations on simple geometric primitives like straight lines and circles. Although existing approaches achieve this by sketch-synthesis-through-optimization, they do so on the space of higher order Bézier curves, which exhibit a wastefully large set of structures that they can evolve into, as most of them are non-essential for generating meaningful sketches. We present CLIPDraw++, an algorithm that provides significantly better visualizations for CLIP text embeddings, using only simple primitive shapes like straight lines and circles. This constrains the set of possible outputs to linear transformations on these primitives, thereby exhibiting an inherently simpler mathematical form. The synthesis process of CLIPDraw++ can be tracked end-to-end, with each visual concept being expressed exclusively in terms of primitives. Project Page: https://clipdrawx.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_02345 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | CLIPDraw++: Text-to-Sketch Synthesis with Simple Primitives Mathur, Nityanand Marjit, Shyam Chaudhuri, Abhra Dutta, Anjan Computer Vision and Pattern Recognition With the goal of understanding the visual concepts that CLIP associates with text prompts, we show that the latent space of CLIP can be visualized solely in terms of linear transformations on simple geometric primitives like straight lines and circles. Although existing approaches achieve this by sketch-synthesis-through-optimization, they do so on the space of higher order Bézier curves, which exhibit a wastefully large set of structures that they can evolve into, as most of them are non-essential for generating meaningful sketches. We present CLIPDraw++, an algorithm that provides significantly better visualizations for CLIP text embeddings, using only simple primitive shapes like straight lines and circles. This constrains the set of possible outputs to linear transformations on these primitives, thereby exhibiting an inherently simpler mathematical form. The synthesis process of CLIPDraw++ can be tracked end-to-end, with each visual concept being expressed exclusively in terms of primitives. Project Page: https://clipdrawx.github.io/. |
| title | CLIPDraw++: Text-to-Sketch Synthesis with Simple Primitives |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2312.02345 |