Exploring Federated Self-Supervised Learning for General Purpose Audio Understanding
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909095040122880 |
|---|---|
| author | Rehman, Yasar Abbas Ur Lau, Kin Wai Xie, Yuyang Ma, Lan Shen, Jiajun |
| author_facet | Rehman, Yasar Abbas Ur Lau, Kin Wai Xie, Yuyang Ma, Lan Shen, Jiajun |
| contents | The integration of Federated Learning (FL) and Self-supervised Learning (SSL) offers a unique and synergetic combination to exploit the audio data for general-purpose audio understanding, without compromising user data privacy. However, rare efforts have been made to investigate the SSL models in the FL regime for general-purpose audio understanding, especially when the training data is generated by large-scale heterogeneous audio sources. In this paper, we evaluate the performance of feature-matching and predictive audio-SSL techniques when integrated into large-scale FL settings simulated with non-independently identically distributed (non-iid) data. We propose a novel Federated SSL (F-SSL) framework, dubbed FASSL, that enables learning intermediate feature representations from large-scale decentralized heterogeneous clients, holding unlabelled audio data. Our study has found that audio F-SSL approaches perform on par with the centralized audio-SSL approaches on the audio-retrieval task. Extensive experiments demonstrate the effectiveness and significance of FASSL as it assists in obtaining the optimal global model for state-of-the-art FL aggregation methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_02889 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Exploring Federated Self-Supervised Learning for General Purpose Audio Understanding Rehman, Yasar Abbas Ur Lau, Kin Wai Xie, Yuyang Ma, Lan Shen, Jiajun Sound Computer Vision and Pattern Recognition Machine Learning Audio and Speech Processing The integration of Federated Learning (FL) and Self-supervised Learning (SSL) offers a unique and synergetic combination to exploit the audio data for general-purpose audio understanding, without compromising user data privacy. However, rare efforts have been made to investigate the SSL models in the FL regime for general-purpose audio understanding, especially when the training data is generated by large-scale heterogeneous audio sources. In this paper, we evaluate the performance of feature-matching and predictive audio-SSL techniques when integrated into large-scale FL settings simulated with non-independently identically distributed (non-iid) data. We propose a novel Federated SSL (F-SSL) framework, dubbed FASSL, that enables learning intermediate feature representations from large-scale decentralized heterogeneous clients, holding unlabelled audio data. Our study has found that audio F-SSL approaches perform on par with the centralized audio-SSL approaches on the audio-retrieval task. Extensive experiments demonstrate the effectiveness and significance of FASSL as it assists in obtaining the optimal global model for state-of-the-art FL aggregation methods. |
| title | Exploring Federated Self-Supervised Learning for General Purpose Audio Understanding |
| topic | Sound Computer Vision and Pattern Recognition Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2402.02889 |