Pattern Analogies: Learning to Perform Programmatic Image Edits by Analogy
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912310128279552 |
|---|---|
| author | Ganeshan, Aditya Groueix, Thibault Guerrero, Paul Měch, Radomír Fisher, Matthew Ritchie, Daniel |
| author_facet | Ganeshan, Aditya Groueix, Thibault Guerrero, Paul Měch, Radomír Fisher, Matthew Ritchie, Daniel |
| contents | Pattern images are everywhere in the digital and physical worlds, and tools to edit them are valuable. But editing pattern images is tricky: desired edits are often programmatic: structure-aware edits that alter the underlying program which generates the pattern. One could attempt to infer this underlying program, but current methods for doing so struggle with complex images and produce unorganized programs that make editing tedious. In this work, we introduce a novel approach to perform programmatic edits on pattern images. By using a pattern analogy -- a pair of simple patterns to demonstrate the intended edit -- and a learning-based generative model to execute these edits, our method allows users to intuitively edit patterns. To enable this paradigm, we introduce SplitWeave, a domain-specific language that, combined with a framework for sampling synthetic pattern analogies, enables the creation of a large, high-quality synthetic training dataset. We also present TriFuser, a Latent Diffusion Model (LDM) designed to overcome critical issues that arise when naively deploying LDMs to this task. Extensive experiments on real-world, artist-sourced patterns reveals that our method faithfully performs the demonstrated edit while also generalizing to related pattern styles beyond its training distribution. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_12463 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Pattern Analogies: Learning to Perform Programmatic Image Edits by Analogy Ganeshan, Aditya Groueix, Thibault Guerrero, Paul Měch, Radomír Fisher, Matthew Ritchie, Daniel Computer Vision and Pattern Recognition Artificial Intelligence Graphics Human-Computer Interaction Pattern images are everywhere in the digital and physical worlds, and tools to edit them are valuable. But editing pattern images is tricky: desired edits are often programmatic: structure-aware edits that alter the underlying program which generates the pattern. One could attempt to infer this underlying program, but current methods for doing so struggle with complex images and produce unorganized programs that make editing tedious. In this work, we introduce a novel approach to perform programmatic edits on pattern images. By using a pattern analogy -- a pair of simple patterns to demonstrate the intended edit -- and a learning-based generative model to execute these edits, our method allows users to intuitively edit patterns. To enable this paradigm, we introduce SplitWeave, a domain-specific language that, combined with a framework for sampling synthetic pattern analogies, enables the creation of a large, high-quality synthetic training dataset. We also present TriFuser, a Latent Diffusion Model (LDM) designed to overcome critical issues that arise when naively deploying LDMs to this task. Extensive experiments on real-world, artist-sourced patterns reveals that our method faithfully performs the demonstrated edit while also generalizing to related pattern styles beyond its training distribution. |
| title | Pattern Analogies: Learning to Perform Programmatic Image Edits by Analogy |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Graphics Human-Computer Interaction |
| url | https://arxiv.org/abs/2412.12463 |