LiteASR: Efficient Automatic Speech Recognition with Low-Rank Approximation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914001375461376 |
|---|---|
| author | Kamahori, Keisuke Kasai, Jungo Kojima, Noriyuki Kasikci, Baris |
| author_facet | Kamahori, Keisuke Kasai, Jungo Kojima, Noriyuki Kasikci, Baris |
| contents | Modern automatic speech recognition (ASR) models, such as OpenAI's Whisper, rely on deep encoder-decoder architectures, and their encoders are a critical bottleneck for efficient deployment due to high computational intensity. We introduce LiteASR, a low-rank compression scheme for ASR encoders that significantly reduces inference costs while maintaining transcription accuracy. Our approach leverages the strong low-rank properties observed in intermediate activations: by applying principal component analysis (PCA) with a small calibration dataset, we approximate linear transformations with a chain of low-rank matrix multiplications, and further optimize self-attention to work in reduced dimensionality. Evaluation results show that our method can compress Whisper large-v3's encoder size by over 50%, matching Whisper medium's size with better transcription accuracy, thereby establishing a new Pareto frontier of accuracy and efficiency. The code of LiteASR is available at https://github.com/efeslab/LiteASR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_20583 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LiteASR: Efficient Automatic Speech Recognition with Low-Rank Approximation Kamahori, Keisuke Kasai, Jungo Kojima, Noriyuki Kasikci, Baris Machine Learning Artificial Intelligence Sound Audio and Speech Processing Modern automatic speech recognition (ASR) models, such as OpenAI's Whisper, rely on deep encoder-decoder architectures, and their encoders are a critical bottleneck for efficient deployment due to high computational intensity. We introduce LiteASR, a low-rank compression scheme for ASR encoders that significantly reduces inference costs while maintaining transcription accuracy. Our approach leverages the strong low-rank properties observed in intermediate activations: by applying principal component analysis (PCA) with a small calibration dataset, we approximate linear transformations with a chain of low-rank matrix multiplications, and further optimize self-attention to work in reduced dimensionality. Evaluation results show that our method can compress Whisper large-v3's encoder size by over 50%, matching Whisper medium's size with better transcription accuracy, thereby establishing a new Pareto frontier of accuracy and efficiency. The code of LiteASR is available at https://github.com/efeslab/LiteASR. |
| title | LiteASR: Efficient Automatic Speech Recognition with Low-Rank Approximation |
| topic | Machine Learning Artificial Intelligence Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2502.20583 |