Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.16026 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911276718882816 |
|---|---|
| author | Al-Qawlaq, Aness M, Ajay Kumar John, Deepu |
| author_facet | Al-Qawlaq, Aness M, Ajay Kumar John, Deepu |
| contents | This paper explores the adaptation of Transformerbased models for edge devices through the quantisation and hardware acceleration of the ARM Keyword Transformer (KWT) model on a RISC-V platform. The model was targeted to run on 64kB RAM in bare-metal C using a custom-developed edge AI library. KWT-1 was retrained to be 369 times smaller, with only a 10% loss in accuracy through reducing output classes from 35 to 2. The retraining and quantisation reduced model size from 2.42 MB to 1.65 kB. The integration of custom RISC-V instructions that accelerated GELU and SoftMax operations enabled a 5x speedup and thus ~5x power reduction in inference, with inference clock cycle counts decreasing from 26 million to 5.5 million clock cycles while incurring a small area overhead of approximately 29%. The results demonstrate a viable method for porting and accelerating Transformer-based models in low-power IoT devices. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_16026 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | KWT-Tiny: RISC-V Accelerated, Embedded Keyword Spotting Transformer Al-Qawlaq, Aness M, Ajay Kumar John, Deepu Hardware Architecture Artificial Intelligence Performance This paper explores the adaptation of Transformerbased models for edge devices through the quantisation and hardware acceleration of the ARM Keyword Transformer (KWT) model on a RISC-V platform. The model was targeted to run on 64kB RAM in bare-metal C using a custom-developed edge AI library. KWT-1 was retrained to be 369 times smaller, with only a 10% loss in accuracy through reducing output classes from 35 to 2. The retraining and quantisation reduced model size from 2.42 MB to 1.65 kB. The integration of custom RISC-V instructions that accelerated GELU and SoftMax operations enabled a 5x speedup and thus ~5x power reduction in inference, with inference clock cycle counts decreasing from 26 million to 5.5 million clock cycles while incurring a small area overhead of approximately 29%. The results demonstrate a viable method for porting and accelerating Transformer-based models in low-power IoT devices. |
| title | KWT-Tiny: RISC-V Accelerated, Embedded Keyword Spotting Transformer |
| topic | Hardware Architecture Artificial Intelligence Performance |
| url | https://arxiv.org/abs/2407.16026 |