Rethink Arbitrary Style Transfer with Transformer and Contrastive Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917646734196736 |
|---|---|
| author | Zhang, Zhanjie Sun, Jiakai Li, Guangyuan Zhao, Lei Zhang, Quanwei Lan, Zehua Yin, Haolin Xing, Wei Lin, Huaizhong Zuo, Zhiwen |
| author_facet | Zhang, Zhanjie Sun, Jiakai Li, Guangyuan Zhao, Lei Zhang, Quanwei Lan, Zehua Yin, Haolin Xing, Wei Lin, Huaizhong Zuo, Zhiwen |
| contents | Arbitrary style transfer holds widespread attention in research and boasts numerous practical applications. The existing methods, which either employ cross-attention to incorporate deep style attributes into content attributes or use adaptive normalization to adjust content features, fail to generate high-quality stylized images. In this paper, we introduce an innovative technique to improve the quality of stylized images. Firstly, we propose Style Consistency Instance Normalization (SCIN), a method to refine the alignment between content and style features. In addition, we have developed an Instance-based Contrastive Learning (ICL) approach designed to understand the relationships among various styles, thereby enhancing the quality of the resulting stylized images. Recognizing that VGG networks are more adept at extracting classification features and need to be better suited for capturing style features, we have also introduced the Perception Encoder (PE) to capture style features. Extensive experiments demonstrate that our proposed method generates high-quality stylized images and effectively prevents artifacts compared with the existing state-of-the-art methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_13584 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Rethink Arbitrary Style Transfer with Transformer and Contrastive Learning Zhang, Zhanjie Sun, Jiakai Li, Guangyuan Zhao, Lei Zhang, Quanwei Lan, Zehua Yin, Haolin Xing, Wei Lin, Huaizhong Zuo, Zhiwen Computer Vision and Pattern Recognition Machine Learning Arbitrary style transfer holds widespread attention in research and boasts numerous practical applications. The existing methods, which either employ cross-attention to incorporate deep style attributes into content attributes or use adaptive normalization to adjust content features, fail to generate high-quality stylized images. In this paper, we introduce an innovative technique to improve the quality of stylized images. Firstly, we propose Style Consistency Instance Normalization (SCIN), a method to refine the alignment between content and style features. In addition, we have developed an Instance-based Contrastive Learning (ICL) approach designed to understand the relationships among various styles, thereby enhancing the quality of the resulting stylized images. Recognizing that VGG networks are more adept at extracting classification features and need to be better suited for capturing style features, we have also introduced the Perception Encoder (PE) to capture style features. Extensive experiments demonstrate that our proposed method generates high-quality stylized images and effectively prevents artifacts compared with the existing state-of-the-art methods. |
| title | Rethink Arbitrary Style Transfer with Transformer and Contrastive Learning |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2404.13584 |