Robust feature knowledge distillation for enhanced performance of lightweight crack segmentation models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909163955683328 |
|---|---|
| author | Chen, Zhaohui Shamsabadi, Elyas Asadi Jiang, Sheng Shen, Luming Dias-da-Costa, Daniel |
| author_facet | Chen, Zhaohui Shamsabadi, Elyas Asadi Jiang, Sheng Shen, Luming Dias-da-Costa, Daniel |
| contents | Vision-based crack detection faces deployment challenges due to the size of robust models and edge device limitations. These can be addressed with lightweight models trained with knowledge distillation (KD). However, state-of-the-art (SOTA) KD methods compromise anti-noise robustness. This paper develops Robust Feature Knowledge Distillation (RFKD), a framework to improve robustness while retaining the precision of light models for crack segmentation. RFKD distils knowledge from a teacher model's logit layers and intermediate feature maps while leveraging mixed clean and noisy images to transfer robust patterns to the student model, improving its precision, generalisation, and anti-noise performance. To validate the proposed RFKD, a lightweight crack segmentation model, PoolingCrack Tiny (PCT), with only 0.5 M parameters, is also designed and used as the student to run the framework. The results show a significant enhancement in noisy images, with RFKD reaching a 62% enhanced mean Dice score (mDS) compared to SOTA KD methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_06258 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Robust feature knowledge distillation for enhanced performance of lightweight crack segmentation models Chen, Zhaohui Shamsabadi, Elyas Asadi Jiang, Sheng Shen, Luming Dias-da-Costa, Daniel Computer Vision and Pattern Recognition Vision-based crack detection faces deployment challenges due to the size of robust models and edge device limitations. These can be addressed with lightweight models trained with knowledge distillation (KD). However, state-of-the-art (SOTA) KD methods compromise anti-noise robustness. This paper develops Robust Feature Knowledge Distillation (RFKD), a framework to improve robustness while retaining the precision of light models for crack segmentation. RFKD distils knowledge from a teacher model's logit layers and intermediate feature maps while leveraging mixed clean and noisy images to transfer robust patterns to the student model, improving its precision, generalisation, and anti-noise performance. To validate the proposed RFKD, a lightweight crack segmentation model, PoolingCrack Tiny (PCT), with only 0.5 M parameters, is also designed and used as the student to run the framework. The results show a significant enhancement in noisy images, with RFKD reaching a 62% enhanced mean Dice score (mDS) compared to SOTA KD methods. |
| title | Robust feature knowledge distillation for enhanced performance of lightweight crack segmentation models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.06258 |