Differentially Private Parameter-Efficient Fine-tuning for Large ASR Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916420926832640 |
|---|---|
| author | Liu, Hongbin Wang, Lun Thakkar, Om Thakurta, Abhradeep Narayanan, Arun |
| author_facet | Liu, Hongbin Wang, Lun Thakkar, Om Thakurta, Abhradeep Narayanan, Arun |
| contents | Large ASR models can inadvertently leak sensitive information, which can be mitigated by formal privacy measures like differential privacy (DP). However, traditional DP training is computationally expensive, and can hurt model performance. Our study explores DP parameter-efficient fine-tuning as a way to mitigate privacy risks with smaller computation and performance costs for ASR models. Through extensive experimentation and progressive optimization, we achieve 4.6%/8.1% word error rate on LibriSpeech clean/other test-sets, setting a new performance benchmark while maintaining (10, 3.52e-6)-DP in fine-tuning a large ASR model with over 600M parameters. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_01948 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Differentially Private Parameter-Efficient Fine-tuning for Large ASR Models Liu, Hongbin Wang, Lun Thakkar, Om Thakurta, Abhradeep Narayanan, Arun Cryptography and Security Large ASR models can inadvertently leak sensitive information, which can be mitigated by formal privacy measures like differential privacy (DP). However, traditional DP training is computationally expensive, and can hurt model performance. Our study explores DP parameter-efficient fine-tuning as a way to mitigate privacy risks with smaller computation and performance costs for ASR models. Through extensive experimentation and progressive optimization, we achieve 4.6%/8.1% word error rate on LibriSpeech clean/other test-sets, setting a new performance benchmark while maintaining (10, 3.52e-6)-DP in fine-tuning a large ASR model with over 600M parameters. |
| title | Differentially Private Parameter-Efficient Fine-tuning for Large ASR Models |
| topic | Cryptography and Security |
| url | https://arxiv.org/abs/2410.01948 |