Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient Pruning of A Multilingual ASR Model
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909651638943744 |
|---|---|
| author | Xie, Jiamin Li, Ke Guo, Jinxi Tjandra, Andros Shangguan, Yuan Sari, Leda Wu, Chunyang Jia, Junteng Mahadeokar, Jay Kalinli, Ozlem |
| author_facet | Xie, Jiamin Li, Ke Guo, Jinxi Tjandra, Andros Shangguan, Yuan Sari, Leda Wu, Chunyang Jia, Junteng Mahadeokar, Jay Kalinli, Ozlem |
| contents | Neural network pruning offers an effective method for compressing a multilingual automatic speech recognition (ASR) model with minimal performance loss. However, it entails several rounds of pruning and re-training needed to be run for each language. In this work, we propose the use of an adaptive masking approach in two scenarios for pruning a multilingual ASR model efficiently, each resulting in sparse monolingual models or a sparse multilingual model (named as Dynamic ASR Pathways). Our approach dynamically adapts the sub-network, avoiding premature decisions about a fixed sub-network structure. We show that our approach outperforms existing pruning methods when targeting sparse monolingual models. Further, we illustrate that Dynamic ASR Pathways jointly discovers and trains better sub-networks (pathways) of a single multilingual model by adapting from different sub-network initializations, thereby reducing the need for language-specific pruning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_13018 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient Pruning of A Multilingual ASR Model Xie, Jiamin Li, Ke Guo, Jinxi Tjandra, Andros Shangguan, Yuan Sari, Leda Wu, Chunyang Jia, Junteng Mahadeokar, Jay Kalinli, Ozlem Audio and Speech Processing Artificial Intelligence Computation and Language Machine Learning Sound Neural network pruning offers an effective method for compressing a multilingual automatic speech recognition (ASR) model with minimal performance loss. However, it entails several rounds of pruning and re-training needed to be run for each language. In this work, we propose the use of an adaptive masking approach in two scenarios for pruning a multilingual ASR model efficiently, each resulting in sparse monolingual models or a sparse multilingual model (named as Dynamic ASR Pathways). Our approach dynamically adapts the sub-network, avoiding premature decisions about a fixed sub-network structure. We show that our approach outperforms existing pruning methods when targeting sparse monolingual models. Further, we illustrate that Dynamic ASR Pathways jointly discovers and trains better sub-networks (pathways) of a single multilingual model by adapting from different sub-network initializations, thereby reducing the need for language-specific pruning. |
| title | Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient Pruning of A Multilingual ASR Model |
| topic | Audio and Speech Processing Artificial Intelligence Computation and Language Machine Learning Sound |
| url | https://arxiv.org/abs/2309.13018 |