Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.02550 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918227968262144 |
|---|---|
| author | Tasou, Ioanna Mpakos, Panagiotis Vlachos, Angelos Adamopoulos, Dionysios Giannakopoulos, Georgios Katsikopoulos, Konstantinos Karaparisis, Ioannis Lazou, Maria Loukovitis, Spyridon Mei, Areti Poulopoulou, Anastasia Dimitriou, Angeliki Filandrianos, Giorgos Galanopoulos, Dimitrios Karampinis, Vasileios Mitsouras, Ilias Spanos, Nikolaos Anastasiadis, Petros Doudalis, Ioannis Nikas, Konstantinos Retsinas, George Tzouveli, Paraskevi Giannoula, Christina Koziris, Nectarios Papadopoulou, Nikela Stamou, Giorgos Voulodimos, Athanasios Goumas, Georgios |
| author_facet | Tasou, Ioanna Mpakos, Panagiotis Vlachos, Angelos Adamopoulos, Dionysios Giannakopoulos, Georgios Katsikopoulos, Konstantinos Karaparisis, Ioannis Lazou, Maria Loukovitis, Spyridon Mei, Areti Poulopoulou, Anastasia Dimitriou, Angeliki Filandrianos, Giorgos Galanopoulos, Dimitrios Karampinis, Vasileios Mitsouras, Ilias Spanos, Nikolaos Anastasiadis, Petros Doudalis, Ioannis Nikas, Konstantinos Retsinas, George Tzouveli, Paraskevi Giannoula, Christina Koziris, Nectarios Papadopoulou, Nikela Stamou, Giorgos Voulodimos, Athanasios Goumas, Georgios |
| contents | The computational demands of modern Deep Neural Networks (DNNs) are immense and constantly growing. While training costs usually capture public attention, inference demands are also contributing in significant computational, energy and environmental footprints. Sparsity stands out as a critical mechanism for drastically reducing these resource demands. However, its potential remains largely untapped and is not yet fully incorporated in production AI systems. To bridge this gap, this work provides the necessary knowledge and insights for performance engineers keen to get involved in deep learning inference optimization. In particular, in this work we: a) discuss the various forms of sparsity that can be utilized in DNN inference, b) explain how the original dense computations translate to sparse kernels, c) provide an extensive bibliographic review of the state-of-the-art in the implementation of these kernels for CPUs and GPUs, d) discuss the availability of sparse datasets in support of sparsity-related research and development, e) explore the current software tools and frameworks that provide robust sparsity support, and f) present evaluation results of different implementations of the key SpMM and SDDMM kernels on CPU and GPU platforms. Ultimately, this paper aims to serve as a resource for performance engineers seeking to develop and deploy highly efficient sparse deep learning models in productions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_02550 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Sparse Computations in Deep Learning Inference Tasou, Ioanna Mpakos, Panagiotis Vlachos, Angelos Adamopoulos, Dionysios Giannakopoulos, Georgios Katsikopoulos, Konstantinos Karaparisis, Ioannis Lazou, Maria Loukovitis, Spyridon Mei, Areti Poulopoulou, Anastasia Dimitriou, Angeliki Filandrianos, Giorgos Galanopoulos, Dimitrios Karampinis, Vasileios Mitsouras, Ilias Spanos, Nikolaos Anastasiadis, Petros Doudalis, Ioannis Nikas, Konstantinos Retsinas, George Tzouveli, Paraskevi Giannoula, Christina Koziris, Nectarios Papadopoulou, Nikela Stamou, Giorgos Voulodimos, Athanasios Goumas, Georgios Computational Engineering, Finance, and Science Artificial Intelligence Machine Learning The computational demands of modern Deep Neural Networks (DNNs) are immense and constantly growing. While training costs usually capture public attention, inference demands are also contributing in significant computational, energy and environmental footprints. Sparsity stands out as a critical mechanism for drastically reducing these resource demands. However, its potential remains largely untapped and is not yet fully incorporated in production AI systems. To bridge this gap, this work provides the necessary knowledge and insights for performance engineers keen to get involved in deep learning inference optimization. In particular, in this work we: a) discuss the various forms of sparsity that can be utilized in DNN inference, b) explain how the original dense computations translate to sparse kernels, c) provide an extensive bibliographic review of the state-of-the-art in the implementation of these kernels for CPUs and GPUs, d) discuss the availability of sparse datasets in support of sparsity-related research and development, e) explore the current software tools and frameworks that provide robust sparsity support, and f) present evaluation results of different implementations of the key SpMM and SDDMM kernels on CPU and GPU platforms. Ultimately, this paper aims to serve as a resource for performance engineers seeking to develop and deploy highly efficient sparse deep learning models in productions. |
| title | Sparse Computations in Deep Learning Inference |
| topic | Computational Engineering, Finance, and Science Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2512.02550 |