Representative Feature Extraction During Diffusion Process for Sketch Extraction with One Example
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929204059176960 |
|---|---|
| author | Yun, Kwan Kim, Youngseo Seo, Kwanggyoon Seo, Chang Wook Noh, Junyong |
| author_facet | Yun, Kwan Kim, Youngseo Seo, Kwanggyoon Seo, Chang Wook Noh, Junyong |
| contents | We introduce DiffSketch, a method for generating a variety of stylized sketches from images. Our approach focuses on selecting representative features from the rich semantics of deep features within a pretrained diffusion model. This novel sketch generation method can be trained with one manual drawing. Furthermore, efficient sketch extraction is ensured by distilling a trained generator into a streamlined extractor. We select denoising diffusion features through analysis and integrate these selected features with VAE features to produce sketches. Additionally, we propose a sampling scheme for training models using a conditional generative approach. Through a series of comparisons, we verify that distilled DiffSketch not only outperforms existing state-of-the-art sketch extraction methods but also surpasses diffusion-based stylization methods in the task of extracting sketches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_04362 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Representative Feature Extraction During Diffusion Process for Sketch Extraction with One Example Yun, Kwan Kim, Youngseo Seo, Kwanggyoon Seo, Chang Wook Noh, Junyong Computer Vision and Pattern Recognition Artificial Intelligence Graphics 68T01 I.4.9 We introduce DiffSketch, a method for generating a variety of stylized sketches from images. Our approach focuses on selecting representative features from the rich semantics of deep features within a pretrained diffusion model. This novel sketch generation method can be trained with one manual drawing. Furthermore, efficient sketch extraction is ensured by distilling a trained generator into a streamlined extractor. We select denoising diffusion features through analysis and integrate these selected features with VAE features to produce sketches. Additionally, we propose a sampling scheme for training models using a conditional generative approach. Through a series of comparisons, we verify that distilled DiffSketch not only outperforms existing state-of-the-art sketch extraction methods but also surpasses diffusion-based stylization methods in the task of extracting sketches. |
| title | Representative Feature Extraction During Diffusion Process for Sketch Extraction with One Example |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Graphics 68T01 I.4.9 |
| url | https://arxiv.org/abs/2401.04362 |