Joint Feature and Output Distillation for Low-complexity Acoustic Scene Classification
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916864780664832 |
|---|---|
| author | Li, Haowen Yang, Ziyi Wang, Mou Tan, Ee-Leng Yeow, Junwei Peksi, Santi Gan, Woon-Seng |
| author_facet | Li, Haowen Yang, Ziyi Wang, Mou Tan, Ee-Leng Yeow, Junwei Peksi, Santi Gan, Woon-Seng |
| contents | This report presents a dual-level knowledge distillation framework with multi-teacher guidance for low-complexity acoustic scene classification (ASC) in DCASE2025 Task 1. We propose a distillation strategy that jointly transfers both soft logits and intermediate feature representations. Specifically, we pre-trained PaSST and CP-ResNet models as teacher models. Logits from teachers are averaged to generate soft targets, while one CP-ResNet is selected for feature-level distillation. This enables the compact student model (CP-Mobile) to capture both semantic distribution and structural information from teacher guidance. Experiments on the TAU Urban Acoustic Scenes 2022 Mobile dataset (development set) demonstrate that our submitted systems achieve up to 59.30\% accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_19557 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Joint Feature and Output Distillation for Low-complexity Acoustic Scene Classification Li, Haowen Yang, Ziyi Wang, Mou Tan, Ee-Leng Yeow, Junwei Peksi, Santi Gan, Woon-Seng Sound Audio and Speech Processing I.2.6 This report presents a dual-level knowledge distillation framework with multi-teacher guidance for low-complexity acoustic scene classification (ASC) in DCASE2025 Task 1. We propose a distillation strategy that jointly transfers both soft logits and intermediate feature representations. Specifically, we pre-trained PaSST and CP-ResNet models as teacher models. Logits from teachers are averaged to generate soft targets, while one CP-ResNet is selected for feature-level distillation. This enables the compact student model (CP-Mobile) to capture both semantic distribution and structural information from teacher guidance. Experiments on the TAU Urban Acoustic Scenes 2022 Mobile dataset (development set) demonstrate that our submitted systems achieve up to 59.30\% accuracy. |
| title | Joint Feature and Output Distillation for Low-complexity Acoustic Scene Classification |
| topic | Sound Audio and Speech Processing I.2.6 |
| url | https://arxiv.org/abs/2507.19557 |