Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.13180 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908412463284224 |
|---|---|
| author | Xu, Jingjing Yang, Zijian Zeyer, Albert Beck, Eugen Schlueter, Ralf Ney, Hermann |
| author_facet | Xu, Jingjing Yang, Zijian Zeyer, Albert Beck, Eugen Schlueter, Ralf Ney, Hermann |
| contents | Architecture design is inherently complex. Existing approaches rely on either handcrafted rules, which demand extensive empirical expertise, or automated methods like neural architecture search, which are computationally intensive. In this paper, we introduce DMAO, an architecture optimization framework that employs a grow-and-drop strategy to automatically reallocate parameters during training. This reallocation shifts resources from less-utilized areas to those parts of the model where they are most beneficial. Notably, DMAO only introduces negligible training overhead at a given model complexity. We evaluate DMAO through experiments with CTC on LibriSpeech, TED-LIUM-v2 and Switchboard datasets. The results show that, using the same amount of training resources, our proposed DMAO consistently improves WER by up to 6% relatively across various architectures, model sizes, and datasets. Furthermore, we analyze the pattern of parameter redistribution and uncover insightful findings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_13180 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dynamic Acoustic Model Architecture Optimization in Training for ASR Xu, Jingjing Yang, Zijian Zeyer, Albert Beck, Eugen Schlueter, Ralf Ney, Hermann Computation and Language Architecture design is inherently complex. Existing approaches rely on either handcrafted rules, which demand extensive empirical expertise, or automated methods like neural architecture search, which are computationally intensive. In this paper, we introduce DMAO, an architecture optimization framework that employs a grow-and-drop strategy to automatically reallocate parameters during training. This reallocation shifts resources from less-utilized areas to those parts of the model where they are most beneficial. Notably, DMAO only introduces negligible training overhead at a given model complexity. We evaluate DMAO through experiments with CTC on LibriSpeech, TED-LIUM-v2 and Switchboard datasets. The results show that, using the same amount of training resources, our proposed DMAO consistently improves WER by up to 6% relatively across various architectures, model sizes, and datasets. Furthermore, we analyze the pattern of parameter redistribution and uncover insightful findings. |
| title | Dynamic Acoustic Model Architecture Optimization in Training for ASR |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2506.13180 |