Systolic Arrays and Structured Pruning Co-design for Efficient Transformers in Edge Systems
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912369105436672 |
|---|---|
| author | Palacios, Pedro Medina, Rafael Rouas, Jean-Luc Ansaloni, Giovanni Atienza, David |
| author_facet | Palacios, Pedro Medina, Rafael Rouas, Jean-Luc Ansaloni, Giovanni Atienza, David |
| contents | Efficient deployment of resource-intensive transformers on edge devices necessitates cross-stack optimization. We thus study the interrelation between structured pruning and systolic acceleration, matching the size of pruned blocks with the systolic array dimensions. In this setting, computations of pruned weight blocks can be skipped, reducing run-time and energy consumption, but potentially impacting quality of service (QoS). To evaluate the trade-offs between systolic array size and sparsity opportunities, we present a novel co-design framework that integrates algorithmic optimization, system simulation, and hardware design. Targeting speech recognition and machine translation using transformers as case study, we analyze how configuration choices across the stack affect performance metrics. Results demonstrate that structured pruning on systems featuring systolic array acceleration can effectively increase performance, while maintaining high QoS levels. Up to 44% system-wide speedups due to structured pruning and quantization were measured, with only 1.4% word error rate degradation on the standard LibriSpeech dataset. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_10285 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Systolic Arrays and Structured Pruning Co-design for Efficient Transformers in Edge Systems Palacios, Pedro Medina, Rafael Rouas, Jean-Luc Ansaloni, Giovanni Atienza, David Hardware Architecture Artificial Intelligence 68T50 C.3; B.5.1; I.2.7 Efficient deployment of resource-intensive transformers on edge devices necessitates cross-stack optimization. We thus study the interrelation between structured pruning and systolic acceleration, matching the size of pruned blocks with the systolic array dimensions. In this setting, computations of pruned weight blocks can be skipped, reducing run-time and energy consumption, but potentially impacting quality of service (QoS). To evaluate the trade-offs between systolic array size and sparsity opportunities, we present a novel co-design framework that integrates algorithmic optimization, system simulation, and hardware design. Targeting speech recognition and machine translation using transformers as case study, we analyze how configuration choices across the stack affect performance metrics. Results demonstrate that structured pruning on systems featuring systolic array acceleration can effectively increase performance, while maintaining high QoS levels. Up to 44% system-wide speedups due to structured pruning and quantization were measured, with only 1.4% word error rate degradation on the standard LibriSpeech dataset. |
| title | Systolic Arrays and Structured Pruning Co-design for Efficient Transformers in Edge Systems |
| topic | Hardware Architecture Artificial Intelligence 68T50 C.3; B.5.1; I.2.7 |
| url | https://arxiv.org/abs/2411.10285 |