Improving Membership Inference in ASR Model Auditing with Perturbed Loss Features
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929333756493824 |
|---|---|
| author | Teixeira, Francisco Pizzi, Karla Olivier, Raphael Abad, Alberto Raj, Bhiksha Trancoso, Isabel |
| author_facet | Teixeira, Francisco Pizzi, Karla Olivier, Raphael Abad, Alberto Raj, Bhiksha Trancoso, Isabel |
| contents | Membership Inference (MI) poses a substantial privacy threat to the training data of Automatic Speech Recognition (ASR) systems, while also offering an opportunity to audit these models with regard to user data. This paper explores the effectiveness of loss-based features in combination with Gaussian and adversarial perturbations to perform MI in ASR models. To the best of our knowledge, this approach has not yet been investigated. We compare our proposed features with commonly used error-based features and find that the proposed features greatly enhance performance for sample-level MI. For speaker-level MI, these features improve results, though by a smaller margin, as error-based features already obtained a high performance for this task. Our findings emphasise the importance of considering different feature sets and levels of access to target models for effective MI in ASR systems, providing valuable insights for auditing such models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_01207 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Improving Membership Inference in ASR Model Auditing with Perturbed Loss Features Teixeira, Francisco Pizzi, Karla Olivier, Raphael Abad, Alberto Raj, Bhiksha Trancoso, Isabel Machine Learning Cryptography and Security Sound Audio and Speech Processing Membership Inference (MI) poses a substantial privacy threat to the training data of Automatic Speech Recognition (ASR) systems, while also offering an opportunity to audit these models with regard to user data. This paper explores the effectiveness of loss-based features in combination with Gaussian and adversarial perturbations to perform MI in ASR models. To the best of our knowledge, this approach has not yet been investigated. We compare our proposed features with commonly used error-based features and find that the proposed features greatly enhance performance for sample-level MI. For speaker-level MI, these features improve results, though by a smaller margin, as error-based features already obtained a high performance for this task. Our findings emphasise the importance of considering different feature sets and levels of access to target models for effective MI in ASR systems, providing valuable insights for auditing such models. |
| title | Improving Membership Inference in ASR Model Auditing with Perturbed Loss Features |
| topic | Machine Learning Cryptography and Security Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2405.01207 |