Effective and Efficient Mixed Precision Quantization of Speech Foundation Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917889485832192 |
|---|---|
| author | Xu, Haoning Li, Zhaoqing Jin, Zengrui Wang, Huimeng Chen, Youjun Li, Guinan Geng, Mengzhe Hu, Shujie Deng, Jiajun Liu, Xunying |
| author_facet | Xu, Haoning Li, Zhaoqing Jin, Zengrui Wang, Huimeng Chen, Youjun Li, Guinan Geng, Mengzhe Hu, Shujie Deng, Jiajun Liu, Xunying |
| contents | This paper presents a novel mixed-precision quantization approach for speech foundation models that tightly integrates mixed-precision learning and quantized model parameter estimation into one single model compression stage. Experiments conducted on LibriSpeech dataset with fine-tuned wav2vec2.0-base and HuBERT-large models suggest the resulting mixed-precision quantized models increased the lossless compression ratio by factors up to 1.7x and 1.9x over the respective uniform-precision and two-stage mixed-precision quantized baselines that perform precision learning and model parameters quantization in separate and disjointed stages, while incurring no statistically word error rate (WER) increase over the 32-bit full-precision models. The system compression time of wav2vec2.0-base and HuBERT-large models is reduced by up to 1.9 and 1.5 times over the two-stage mixed-precision baselines, while both produce lower WERs. The best-performing 3.5-bit mixed-precision quantized HuBERT-large model produces a lossless compression ratio of 8.6x over the 32-bit full-precision system. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_03643 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Effective and Efficient Mixed Precision Quantization of Speech Foundation Models Xu, Haoning Li, Zhaoqing Jin, Zengrui Wang, Huimeng Chen, Youjun Li, Guinan Geng, Mengzhe Hu, Shujie Deng, Jiajun Liu, Xunying Sound Artificial Intelligence Audio and Speech Processing This paper presents a novel mixed-precision quantization approach for speech foundation models that tightly integrates mixed-precision learning and quantized model parameter estimation into one single model compression stage. Experiments conducted on LibriSpeech dataset with fine-tuned wav2vec2.0-base and HuBERT-large models suggest the resulting mixed-precision quantized models increased the lossless compression ratio by factors up to 1.7x and 1.9x over the respective uniform-precision and two-stage mixed-precision quantized baselines that perform precision learning and model parameters quantization in separate and disjointed stages, while incurring no statistically word error rate (WER) increase over the 32-bit full-precision models. The system compression time of wav2vec2.0-base and HuBERT-large models is reduced by up to 1.9 and 1.5 times over the two-stage mixed-precision baselines, while both produce lower WERs. The best-performing 3.5-bit mixed-precision quantized HuBERT-large model produces a lossless compression ratio of 8.6x over the 32-bit full-precision system. |
| title | Effective and Efficient Mixed Precision Quantization of Speech Foundation Models |
| topic | Sound Artificial Intelligence Audio and Speech Processing |
| url | https://arxiv.org/abs/2501.03643 |