Nearly Zero-Cost Protection Against Mimicry by Personalized Diffusion Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917983452921856 |
|---|---|
| author | Ahn, Namhyuk Yoo, KiYoon Ahn, Wonhyuk Kim, Daesik Nam, Seung-Hun |
| author_facet | Ahn, Namhyuk Yoo, KiYoon Ahn, Wonhyuk Kim, Daesik Nam, Seung-Hun |
| contents | Recent advancements in diffusion models revolutionize image generation but pose risks of misuse, such as replicating artworks or generating deepfakes. Existing image protection methods, though effective, struggle to balance protection efficacy, invisibility, and latency, thus limiting practical use. We introduce perturbation pre-training to reduce latency and propose a mixture-of-perturbations approach that dynamically adapts to input images to minimize performance degradation. Our novel training strategy computes protection loss across multiple VAE feature spaces, while adaptive targeted protection at inference enhances robustness and invisibility. Experiments show comparable protection performance with improved invisibility and drastically reduced inference time. The code and demo are available at https://webtoon.github.io/impasto |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_11423 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Nearly Zero-Cost Protection Against Mimicry by Personalized Diffusion Models Ahn, Namhyuk Yoo, KiYoon Ahn, Wonhyuk Kim, Daesik Nam, Seung-Hun Computer Vision and Pattern Recognition Recent advancements in diffusion models revolutionize image generation but pose risks of misuse, such as replicating artworks or generating deepfakes. Existing image protection methods, though effective, struggle to balance protection efficacy, invisibility, and latency, thus limiting practical use. We introduce perturbation pre-training to reduce latency and propose a mixture-of-perturbations approach that dynamically adapts to input images to minimize performance degradation. Our novel training strategy computes protection loss across multiple VAE feature spaces, while adaptive targeted protection at inference enhances robustness and invisibility. Experiments show comparable protection performance with improved invisibility and drastically reduced inference time. The code and demo are available at https://webtoon.github.io/impasto |
| title | Nearly Zero-Cost Protection Against Mimicry by Personalized Diffusion Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.11423 |