IKD+: Reliable Low Complexity Deep Models For Retinopathy Classification
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913232215605248 |
|---|---|
| author | Brahmavar, Shreyas Bhat Rajesh, Rohit Dash, Tirtharaj Vig, Lovekesh Verlekar, Tanmay Tulsidas Hasan, Md Mahmudul Khan, Tariq Meijering, Erik Srinivasan, Ashwin |
| author_facet | Brahmavar, Shreyas Bhat Rajesh, Rohit Dash, Tirtharaj Vig, Lovekesh Verlekar, Tanmay Tulsidas Hasan, Md Mahmudul Khan, Tariq Meijering, Erik Srinivasan, Ashwin |
| contents | Deep neural network (DNN) models for retinopathy have estimated predictive accuracies in the mid-to-high 90%. However, the following aspects remain unaddressed: State-of-the-art models are complex and require substantial computational infrastructure to train and deploy; The reliability of predictions can vary widely. In this paper, we focus on these aspects and propose a form of iterative knowledge distillation(IKD), called IKD+ that incorporates a tradeoff between size, accuracy and reliability. We investigate the functioning of IKD+ using two widely used techniques for estimating model calibration (Platt-scaling and temperature-scaling), using the best-performing model available, which is an ensemble of EfficientNets with approximately 100M parameters. We demonstrate that IKD+ equipped with temperature-scaling results in models that show up to approximately 500-fold decreases in the number of parameters than the original ensemble without a significant loss in accuracy. In addition, calibration scores (reliability) for the IKD+ models are as good as or better than the base mode |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2303_02310 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | IKD+: Reliable Low Complexity Deep Models For Retinopathy Classification Brahmavar, Shreyas Bhat Rajesh, Rohit Dash, Tirtharaj Vig, Lovekesh Verlekar, Tanmay Tulsidas Hasan, Md Mahmudul Khan, Tariq Meijering, Erik Srinivasan, Ashwin Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Deep neural network (DNN) models for retinopathy have estimated predictive accuracies in the mid-to-high 90%. However, the following aspects remain unaddressed: State-of-the-art models are complex and require substantial computational infrastructure to train and deploy; The reliability of predictions can vary widely. In this paper, we focus on these aspects and propose a form of iterative knowledge distillation(IKD), called IKD+ that incorporates a tradeoff between size, accuracy and reliability. We investigate the functioning of IKD+ using two widely used techniques for estimating model calibration (Platt-scaling and temperature-scaling), using the best-performing model available, which is an ensemble of EfficientNets with approximately 100M parameters. We demonstrate that IKD+ equipped with temperature-scaling results in models that show up to approximately 500-fold decreases in the number of parameters than the original ensemble without a significant loss in accuracy. In addition, calibration scores (reliability) for the IKD+ models are as good as or better than the base mode |
| title | IKD+: Reliable Low Complexity Deep Models For Retinopathy Classification |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2303.02310 |