Nerva: a Truly Sparse Implementation of Neural Networks
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917732151197696 |
|---|---|
| author | Wesselink, Wieger Grooten, Bram Xiao, Qiao de Campos, Cassio Pechenizkiy, Mykola |
| author_facet | Wesselink, Wieger Grooten, Bram Xiao, Qiao de Campos, Cassio Pechenizkiy, Mykola |
| contents | We introduce Nerva, a fast neural network library under development in C++. It supports sparsity by using the sparse matrix operations of Intel's Math Kernel Library (MKL), which eliminates the need for binary masks. We show that Nerva significantly decreases training time and memory usage while reaching equivalent accuracy to PyTorch. We run static sparse experiments with an MLP on CIFAR-10. On high sparsity levels like $99\%$, the runtime is reduced by a factor of $4\times$ compared to a PyTorch model using masks. Similar to other popular frameworks such as PyTorch and Keras, Nerva offers a Python interface for users to work with. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_17437 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Nerva: a Truly Sparse Implementation of Neural Networks Wesselink, Wieger Grooten, Bram Xiao, Qiao de Campos, Cassio Pechenizkiy, Mykola Machine Learning We introduce Nerva, a fast neural network library under development in C++. It supports sparsity by using the sparse matrix operations of Intel's Math Kernel Library (MKL), which eliminates the need for binary masks. We show that Nerva significantly decreases training time and memory usage while reaching equivalent accuracy to PyTorch. We run static sparse experiments with an MLP on CIFAR-10. On high sparsity levels like $99\%$, the runtime is reduced by a factor of $4\times$ compared to a PyTorch model using masks. Similar to other popular frameworks such as PyTorch and Keras, Nerva offers a Python interface for users to work with. |
| title | Nerva: a Truly Sparse Implementation of Neural Networks |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2407.17437 |