Toward Attention-based TinyML: A Heterogeneous Accelerated Architecture and Automated Deployment Flow
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929666899574784 |
|---|---|
| author | Wiese, Philip İslamoğlu, Gamze Scherer, Moritz Macan, Luka Jung, Victor J. B. Burrello, Alessio Conti, Francesco Benini, Luca |
| author_facet | Wiese, Philip İslamoğlu, Gamze Scherer, Moritz Macan, Luka Jung, Victor J. B. Burrello, Alessio Conti, Francesco Benini, Luca |
| contents | One of the challenges for Tiny Machine Learning (tinyML) is keeping up with the evolution of Machine Learning models from Convolutional Neural Networks to Transformers. We address this by leveraging a heterogeneous architectural template coupling RISC-V processors with hardwired accelerators supported by an automated deployment flow. We demonstrate Attention-based models in a tinyML power envelope with an octa-core cluster coupled with an accelerator for quantized Attention. Our deployment flow enables end-to-end 8-bit Transformer inference, achieving leading-edge energy efficiency and throughput of 2960 GOp/J and 154 GOp/s (0.65 V, 22 nm FD-SOI technology). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_02473 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Toward Attention-based TinyML: A Heterogeneous Accelerated Architecture and Automated Deployment Flow Wiese, Philip İslamoğlu, Gamze Scherer, Moritz Macan, Luka Jung, Victor J. B. Burrello, Alessio Conti, Francesco Benini, Luca Hardware Architecture Machine Learning One of the challenges for Tiny Machine Learning (tinyML) is keeping up with the evolution of Machine Learning models from Convolutional Neural Networks to Transformers. We address this by leveraging a heterogeneous architectural template coupling RISC-V processors with hardwired accelerators supported by an automated deployment flow. We demonstrate Attention-based models in a tinyML power envelope with an octa-core cluster coupled with an accelerator for quantized Attention. Our deployment flow enables end-to-end 8-bit Transformer inference, achieving leading-edge energy efficiency and throughput of 2960 GOp/J and 154 GOp/s (0.65 V, 22 nm FD-SOI technology). |
| title | Toward Attention-based TinyML: A Heterogeneous Accelerated Architecture and Automated Deployment Flow |
| topic | Hardware Architecture Machine Learning |
| url | https://arxiv.org/abs/2408.02473 |