E$^{2}$GAN: Efficient Training of Efficient GANs for Image-to-Image Translation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916270290501632 |
|---|---|
| author | Gong, Yifan Zhan, Zheng Jin, Qing Li, Yanyu Idelbayev, Yerlan Liu, Xian Zharkov, Andrey Aberman, Kfir Tulyakov, Sergey Wang, Yanzhi Ren, Jian |
| author_facet | Gong, Yifan Zhan, Zheng Jin, Qing Li, Yanyu Idelbayev, Yerlan Liu, Xian Zharkov, Andrey Aberman, Kfir Tulyakov, Sergey Wang, Yanzhi Ren, Jian |
| contents | One highly promising direction for enabling flexible real-time on-device image editing is utilizing data distillation by leveraging large-scale text-to-image diffusion models to generate paired datasets used for training generative adversarial networks (GANs). This approach notably alleviates the stringent requirements typically imposed by high-end commercial GPUs for performing image editing with diffusion models. However, unlike text-to-image diffusion models, each distilled GAN is specialized for a specific image editing task, necessitating costly training efforts to obtain models for various concepts. In this work, we introduce and address a novel research direction: can the process of distilling GANs from diffusion models be made significantly more efficient? To achieve this goal, we propose a series of innovative techniques. First, we construct a base GAN model with generalized features, adaptable to different concepts through fine-tuning, eliminating the need for training from scratch. Second, we identify crucial layers within the base GAN model and employ Low-Rank Adaptation (LoRA) with a simple yet effective rank search process, rather than fine-tuning the entire base model. Third, we investigate the minimal amount of data necessary for fine-tuning, further reducing the overall training time. Extensive experiments show that we can efficiently empower GANs with the ability to perform real-time high-quality image editing on mobile devices with remarkably reduced training and storage costs for each concept. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_06127 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | E$^{2}$GAN: Efficient Training of Efficient GANs for Image-to-Image Translation Gong, Yifan Zhan, Zheng Jin, Qing Li, Yanyu Idelbayev, Yerlan Liu, Xian Zharkov, Andrey Aberman, Kfir Tulyakov, Sergey Wang, Yanzhi Ren, Jian Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning One highly promising direction for enabling flexible real-time on-device image editing is utilizing data distillation by leveraging large-scale text-to-image diffusion models to generate paired datasets used for training generative adversarial networks (GANs). This approach notably alleviates the stringent requirements typically imposed by high-end commercial GPUs for performing image editing with diffusion models. However, unlike text-to-image diffusion models, each distilled GAN is specialized for a specific image editing task, necessitating costly training efforts to obtain models for various concepts. In this work, we introduce and address a novel research direction: can the process of distilling GANs from diffusion models be made significantly more efficient? To achieve this goal, we propose a series of innovative techniques. First, we construct a base GAN model with generalized features, adaptable to different concepts through fine-tuning, eliminating the need for training from scratch. Second, we identify crucial layers within the base GAN model and employ Low-Rank Adaptation (LoRA) with a simple yet effective rank search process, rather than fine-tuning the entire base model. Third, we investigate the minimal amount of data necessary for fine-tuning, further reducing the overall training time. Extensive experiments show that we can efficiently empower GANs with the ability to perform real-time high-quality image editing on mobile devices with remarkably reduced training and storage costs for each concept. |
| title | E$^{2}$GAN: Efficient Training of Efficient GANs for Image-to-Image Translation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2401.06127 |