CtlGAN: Few-shot Artistic Portraits Generation with Contrastive Transfer Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2022
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911791088402432 |
|---|---|
| author | Wang, Yue Yi, Ran Li, Luying Tai, Ying Wang, Chengjie Ma, Lizhuang |
| author_facet | Wang, Yue Yi, Ran Li, Luying Tai, Ying Wang, Chengjie Ma, Lizhuang |
| contents | Generating artistic portraits is a challenging problem in computer vision. Existing portrait stylization models that generate good quality results are based on Image-to-Image Translation and require abundant data from both source and target domains. However, without enough data, these methods would result in overfitting. In this work, we propose CtlGAN, a new few-shot artistic portraits generation model with a novel contrastive transfer learning strategy. We adapt a pretrained StyleGAN in the source domain to a target artistic domain with no more than 10 artistic faces. To reduce overfitting to the few training examples, we introduce a novel Cross-Domain Triplet loss which explicitly encourages the target instances generated from different latent codes to be distinguishable. We propose a new encoder which embeds real faces into Z+ space and proposes a dual-path training strategy to better cope with the adapted decoder and eliminate the artifacts. Extensive qualitative, quantitative comparisons and a user study show our method significantly outperforms state-of-the-arts under 10-shot and 1-shot settings and generates high quality artistic portraits. The code will be made publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2203_08612 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | CtlGAN: Few-shot Artistic Portraits Generation with Contrastive Transfer Learning Wang, Yue Yi, Ran Li, Luying Tai, Ying Wang, Chengjie Ma, Lizhuang Computer Vision and Pattern Recognition Graphics Generating artistic portraits is a challenging problem in computer vision. Existing portrait stylization models that generate good quality results are based on Image-to-Image Translation and require abundant data from both source and target domains. However, without enough data, these methods would result in overfitting. In this work, we propose CtlGAN, a new few-shot artistic portraits generation model with a novel contrastive transfer learning strategy. We adapt a pretrained StyleGAN in the source domain to a target artistic domain with no more than 10 artistic faces. To reduce overfitting to the few training examples, we introduce a novel Cross-Domain Triplet loss which explicitly encourages the target instances generated from different latent codes to be distinguishable. We propose a new encoder which embeds real faces into Z+ space and proposes a dual-path training strategy to better cope with the adapted decoder and eliminate the artifacts. Extensive qualitative, quantitative comparisons and a user study show our method significantly outperforms state-of-the-arts under 10-shot and 1-shot settings and generates high quality artistic portraits. The code will be made publicly available. |
| title | CtlGAN: Few-shot Artistic Portraits Generation with Contrastive Transfer Learning |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2203.08612 |