Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913297415012352 |
|---|---|
| author | Li, Simiao Zhang, Yun Li, Wei Chen, Hanting Wang, Wenjia Jing, Bingyi Lin, Shaohui Hu, Jie |
| author_facet | Li, Simiao Zhang, Yun Li, Wei Chen, Hanting Wang, Wenjia Jing, Bingyi Lin, Shaohui Hu, Jie |
| contents | Knowledge distillation (KD) is a promising yet challenging model compression technique that transfers rich learning representations from a well-performing but cumbersome teacher model to a compact student model. Previous methods for image super-resolution (SR) mostly compare the feature maps directly or after standardizing the dimensions with basic algebraic operations (e.g. average, dot-product). However, the intrinsic semantic differences among feature maps are overlooked, which are caused by the disparate expressive capacity between the networks. This work presents MiPKD, a multi-granularity mixture of prior KD framework, to facilitate efficient SR model through the feature mixture in a unified latent space and stochastic network block mixture. Extensive experiments demonstrate the effectiveness of the proposed MiPKD method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_02573 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution Li, Simiao Zhang, Yun Li, Wei Chen, Hanting Wang, Wenjia Jing, Bingyi Lin, Shaohui Hu, Jie Computer Vision and Pattern Recognition Knowledge distillation (KD) is a promising yet challenging model compression technique that transfers rich learning representations from a well-performing but cumbersome teacher model to a compact student model. Previous methods for image super-resolution (SR) mostly compare the feature maps directly or after standardizing the dimensions with basic algebraic operations (e.g. average, dot-product). However, the intrinsic semantic differences among feature maps are overlooked, which are caused by the disparate expressive capacity between the networks. This work presents MiPKD, a multi-granularity mixture of prior KD framework, to facilitate efficient SR model through the feature mixture in a unified latent space and stochastic network block mixture. Extensive experiments demonstrate the effectiveness of the proposed MiPKD method. |
| title | Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.02573 |