Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2312.10359 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917664876658688 |
|---|---|
| author | Xu, Mingbin Jin, Alex Wang, Sicheng Su, Mu Ng, Tim Mason, Henry Han, Shiyi Lei, Zhihong Deng, Yaqiao Huang, Zhen Krishnamoorthy, Mahesh |
| author_facet | Xu, Mingbin Jin, Alex Wang, Sicheng Su, Mu Ng, Tim Mason, Henry Han, Shiyi Lei, Zhihong Deng, Yaqiao Huang, Zhen Krishnamoorthy, Mahesh |
| contents | With increasingly more powerful compute capabilities and resources in today's devices, traditionally compute-intensive automatic speech recognition (ASR) has been moving from the cloud to devices to better protect user privacy. However, it is still challenging to implement on-device ASR on resource-constrained devices, such as smartphones, smart wearables, and other smart home automation devices. In this paper, we propose a series of model architecture adaptions, neural network graph transformations, and numerical optimizations to fit an advanced Conformer based end-to-end streaming ASR system on resource-constrained devices without accuracy degradation. We achieve over 5.26 times faster than realtime (0.19 RTF) speech recognition on smart wearables while minimizing energy consumption and achieving state-of-the-art accuracy. The proposed methods are widely applicable to other transformer-based server-free AI applications. In addition, we provide a complete theory on optimal pre-normalizers that numerically stabilize layer normalization in any Lp-norm using any floating point precision. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_10359 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Conformer-Based Speech Recognition On Extreme Edge-Computing Devices Xu, Mingbin Jin, Alex Wang, Sicheng Su, Mu Ng, Tim Mason, Henry Han, Shiyi Lei, Zhihong Deng, Yaqiao Huang, Zhen Krishnamoorthy, Mahesh Machine Learning Performance With increasingly more powerful compute capabilities and resources in today's devices, traditionally compute-intensive automatic speech recognition (ASR) has been moving from the cloud to devices to better protect user privacy. However, it is still challenging to implement on-device ASR on resource-constrained devices, such as smartphones, smart wearables, and other smart home automation devices. In this paper, we propose a series of model architecture adaptions, neural network graph transformations, and numerical optimizations to fit an advanced Conformer based end-to-end streaming ASR system on resource-constrained devices without accuracy degradation. We achieve over 5.26 times faster than realtime (0.19 RTF) speech recognition on smart wearables while minimizing energy consumption and achieving state-of-the-art accuracy. The proposed methods are widely applicable to other transformer-based server-free AI applications. In addition, we provide a complete theory on optimal pre-normalizers that numerically stabilize layer normalization in any Lp-norm using any floating point precision. |
| title | Conformer-Based Speech Recognition On Extreme Edge-Computing Devices |
| topic | Machine Learning Performance |
| url | https://arxiv.org/abs/2312.10359 |