WeCKD: Weakly-supervised Chained Distillation Network for Efficient Multimodal Medical Imaging
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911247830614016 |
|---|---|
| author | Rahman, Md. Abdur Raiaan, Mohaimenul Azam Khan Azam, Sami Karim, Asif Beissbarth, Jemima Leach, Amanda |
| author_facet | Rahman, Md. Abdur Raiaan, Mohaimenul Azam Khan Azam, Sami Karim, Asif Beissbarth, Jemima Leach, Amanda |
| contents | Knowledge distillation (KD) has traditionally relied on a static teacher-student framework, where a large, well-trained teacher transfers knowledge to a single student model. However, these approaches often suffer from knowledge degradation, inefficient supervision, and reliance on either a very strong teacher model or large labeled datasets. To address these, we present the first-ever Weakly-supervised Chain-based KD network (WeCKD) that redefines knowledge transfer through a structured sequence of interconnected models. Unlike conventional KD, it forms a progressive distillation chain, where each model not only learns from its predecessor but also refines the knowledge before passing it forward. This structured knowledge transfer further enhances feature learning and addresses the limitations of one-step KD. Each model in the chain is trained on only a fraction of the dataset and shows that effective learning can be achieved with minimal supervision. Extensive evaluation on six imaging datasets across otoscopic, microscopic, and magnetic resonance imaging modalities shows that it generalizes and outperforms existing methods. Furthermore, the proposed distillation chain resulted in cumulative accuracy gains of up to +23% over a single backbone trained on the same limited data, which highlights its potential for real-world adoption. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_14668 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | WeCKD: Weakly-supervised Chained Distillation Network for Efficient Multimodal Medical Imaging Rahman, Md. Abdur Raiaan, Mohaimenul Azam Khan Azam, Sami Karim, Asif Beissbarth, Jemima Leach, Amanda Computer Vision and Pattern Recognition Knowledge distillation (KD) has traditionally relied on a static teacher-student framework, where a large, well-trained teacher transfers knowledge to a single student model. However, these approaches often suffer from knowledge degradation, inefficient supervision, and reliance on either a very strong teacher model or large labeled datasets. To address these, we present the first-ever Weakly-supervised Chain-based KD network (WeCKD) that redefines knowledge transfer through a structured sequence of interconnected models. Unlike conventional KD, it forms a progressive distillation chain, where each model not only learns from its predecessor but also refines the knowledge before passing it forward. This structured knowledge transfer further enhances feature learning and addresses the limitations of one-step KD. Each model in the chain is trained on only a fraction of the dataset and shows that effective learning can be achieved with minimal supervision. Extensive evaluation on six imaging datasets across otoscopic, microscopic, and magnetic resonance imaging modalities shows that it generalizes and outperforms existing methods. Furthermore, the proposed distillation chain resulted in cumulative accuracy gains of up to +23% over a single backbone trained on the same limited data, which highlights its potential for real-world adoption. |
| title | WeCKD: Weakly-supervised Chained Distillation Network for Efficient Multimodal Medical Imaging |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.14668 |