MST-KD: Multiple Specialized Teachers Knowledge Distillation for Fair Face Recognition
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913485526401024 |
|---|---|
| author | Caldeira, Eduarda Cardoso, Jaime S. Sequeira, Ana F. Neto, Pedro C. |
| author_facet | Caldeira, Eduarda Cardoso, Jaime S. Sequeira, Ana F. Neto, Pedro C. |
| contents | As in school, one teacher to cover all subjects is insufficient to distill equally robust information to a student. Hence, each subject is taught by a highly specialised teacher. Following a similar philosophy, we propose a multiple specialized teacher framework to distill knowledge to a student network. In our approach, directed at face recognition use cases, we train four teachers on one specific ethnicity, leading to four highly specialized and biased teachers. Our strategy learns a project of these four teachers into a common space and distill that information to a student network. Our results highlighted increased performance and reduced bias for all our experiments. In addition, we further show that having biased/specialized teachers is crucial by showing that our approach achieves better results than when knowledge is distilled from four teachers trained on balanced datasets. Our approach represents a step forward to the understanding of the importance of ethnicity-specific features. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_16563 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MST-KD: Multiple Specialized Teachers Knowledge Distillation for Fair Face Recognition Caldeira, Eduarda Cardoso, Jaime S. Sequeira, Ana F. Neto, Pedro C. Computer Vision and Pattern Recognition As in school, one teacher to cover all subjects is insufficient to distill equally robust information to a student. Hence, each subject is taught by a highly specialised teacher. Following a similar philosophy, we propose a multiple specialized teacher framework to distill knowledge to a student network. In our approach, directed at face recognition use cases, we train four teachers on one specific ethnicity, leading to four highly specialized and biased teachers. Our strategy learns a project of these four teachers into a common space and distill that information to a student network. Our results highlighted increased performance and reduced bias for all our experiments. In addition, we further show that having biased/specialized teachers is crucial by showing that our approach achieves better results than when knowledge is distilled from four teachers trained on balanced datasets. Our approach represents a step forward to the understanding of the importance of ethnicity-specific features. |
| title | MST-KD: Multiple Specialized Teachers Knowledge Distillation for Fair Face Recognition |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.16563 |