Accelerating Transformer Inference and Training with 2:4 Activation Sparsity
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910888113471488 |
|---|---|
| author | Haziza, Daniel Chou, Timothy Choudhary, Dhruv Wehrstedt, Luca Massa, Francisco Yu, Jiecao Jeong, Geonhwa Rao, Supriya Labatut, Patrick Cai, Jesse |
| author_facet | Haziza, Daniel Chou, Timothy Choudhary, Dhruv Wehrstedt, Luca Massa, Francisco Yu, Jiecao Jeong, Geonhwa Rao, Supriya Labatut, Patrick Cai, Jesse |
| contents | In this paper, we demonstrate how to leverage 2:4 sparsity, a popular hardware-accelerated GPU sparsity pattern, to activations to accelerate large language model training and inference. Crucially we exploit the intrinsic sparsity found in Squared-ReLU activations to provide this acceleration with no accuracy loss. Our approach achieves up to 1.3x faster Feed Forward Network (FFNs) in both the forwards and backwards pass. This work highlights the potential for sparsity to play a key role in accelerating large language model training and inference. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_16672 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Accelerating Transformer Inference and Training with 2:4 Activation Sparsity Haziza, Daniel Chou, Timothy Choudhary, Dhruv Wehrstedt, Luca Massa, Francisco Yu, Jiecao Jeong, Geonhwa Rao, Supriya Labatut, Patrick Cai, Jesse Machine Learning Artificial Intelligence I.2 In this paper, we demonstrate how to leverage 2:4 sparsity, a popular hardware-accelerated GPU sparsity pattern, to activations to accelerate large language model training and inference. Crucially we exploit the intrinsic sparsity found in Squared-ReLU activations to provide this acceleration with no accuracy loss. Our approach achieves up to 1.3x faster Feed Forward Network (FFNs) in both the forwards and backwards pass. This work highlights the potential for sparsity to play a key role in accelerating large language model training and inference. |
| title | Accelerating Transformer Inference and Training with 2:4 Activation Sparsity |
| topic | Machine Learning Artificial Intelligence I.2 |
| url | https://arxiv.org/abs/2503.16672 |