A Novel Garment Transfer Method Supervised by Distilled Knowledge of Virtual Try-on Model
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913298095538176 |
|---|---|
| author | Fang, Naiyu Qiu, Lemiao Zhang, Shuyou Wang, Zili Hu, Kerui Tan, Jianrong |
| author_facet | Fang, Naiyu Qiu, Lemiao Zhang, Shuyou Wang, Zili Hu, Kerui Tan, Jianrong |
| contents | This paper proposes a novel garment transfer method supervised with knowledge distillation from virtual try-on. Our method first reasons the transfer parsing to provide shape prior to downstream tasks. We employ a multi-phase teaching strategy to supervise the training of the transfer parsing reasoning model, learning the response and feature knowledge from the try-on parsing reasoning model. To correct the teaching error, it transfers the garment back to its owner to absorb the hard knowledge in the self-study phase. Guided by the transfer parsing, we adjust the position of the transferred garment via STN to prevent distortion. Afterward, we estimate a progressive flow to precisely warp the garment with shape and content correspondences. To ensure warping rationality, we supervise the training of the garment warping model using target shape and warping knowledge from virtual try-on. To better preserve body features in the transfer result, we propose a well-designed training strategy for the arm regrowth task to infer new exposure skin. Experiments demonstrate that our method has state-of-the-art performance compared with other virtual try-on and garment transfer methods in garment transfer, especially for preserving garment texture and body features. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_12433 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Novel Garment Transfer Method Supervised by Distilled Knowledge of Virtual Try-on Model Fang, Naiyu Qiu, Lemiao Zhang, Shuyou Wang, Zili Hu, Kerui Tan, Jianrong Computer Vision and Pattern Recognition This paper proposes a novel garment transfer method supervised with knowledge distillation from virtual try-on. Our method first reasons the transfer parsing to provide shape prior to downstream tasks. We employ a multi-phase teaching strategy to supervise the training of the transfer parsing reasoning model, learning the response and feature knowledge from the try-on parsing reasoning model. To correct the teaching error, it transfers the garment back to its owner to absorb the hard knowledge in the self-study phase. Guided by the transfer parsing, we adjust the position of the transferred garment via STN to prevent distortion. Afterward, we estimate a progressive flow to precisely warp the garment with shape and content correspondences. To ensure warping rationality, we supervise the training of the garment warping model using target shape and warping knowledge from virtual try-on. To better preserve body features in the transfer result, we propose a well-designed training strategy for the arm regrowth task to infer new exposure skin. Experiments demonstrate that our method has state-of-the-art performance compared with other virtual try-on and garment transfer methods in garment transfer, especially for preserving garment texture and body features. |
| title | A Novel Garment Transfer Method Supervised by Distilled Knowledge of Virtual Try-on Model |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2401.12433 |