CTC Blank Triggered Dynamic Layer-Skipping for Efficient CTC-based Speech Recognition
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911747319791616 |
|---|---|
| author | Hou, Junfeng Wang, Peiyao Zhang, Jincheng Yang, Meng Feng, Minwei Yin, Jingcheng |
| author_facet | Hou, Junfeng Wang, Peiyao Zhang, Jincheng Yang, Meng Feng, Minwei Yin, Jingcheng |
| contents | Deploying end-to-end speech recognition models with limited computing resources remains challenging, despite their impressive performance. Given the gradual increase in model size and the wide range of model applications, selectively executing model components for different inputs to improve the inference efficiency is of great interest. In this paper, we propose a dynamic layer-skipping method that leverages the CTC blank output from intermediate layers to trigger the skipping of the last few encoder layers for frames with high blank probabilities. Furthermore, we factorize the CTC output distribution and perform knowledge distillation on intermediate layers to reduce computation and improve recognition accuracy. Experimental results show that by utilizing the CTC blank, the encoder layer depth can be adjusted dynamically, resulting in 29% acceleration of the CTC model inference with minor performance degradation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_02046 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CTC Blank Triggered Dynamic Layer-Skipping for Efficient CTC-based Speech Recognition Hou, Junfeng Wang, Peiyao Zhang, Jincheng Yang, Meng Feng, Minwei Yin, Jingcheng Audio and Speech Processing Sound Deploying end-to-end speech recognition models with limited computing resources remains challenging, despite their impressive performance. Given the gradual increase in model size and the wide range of model applications, selectively executing model components for different inputs to improve the inference efficiency is of great interest. In this paper, we propose a dynamic layer-skipping method that leverages the CTC blank output from intermediate layers to trigger the skipping of the last few encoder layers for frames with high blank probabilities. Furthermore, we factorize the CTC output distribution and perform knowledge distillation on intermediate layers to reduce computation and improve recognition accuracy. Experimental results show that by utilizing the CTC blank, the encoder layer depth can be adjusted dynamically, resulting in 29% acceleration of the CTC model inference with minor performance degradation. |
| title | CTC Blank Triggered Dynamic Layer-Skipping for Efficient CTC-based Speech Recognition |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2401.02046 |