Cardinality Sparsity: Applications in Matrix-Matrix Multiplications and Machine Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911130593525760 |
|---|---|
| author | Mohades, Ali Lederer, Johannes |
| author_facet | Mohades, Ali Lederer, Johannes |
| contents | High-dimensional data has become ubiquitous across the sciences but presents computational and statistical challenges. A common approach to addressing these challenges is through sparsity. In this paper, we introduce a new concept of sparsity, called cardinality sparsity. Broadly speaking, we define a tensor as sparse if it contains only a small number of unique values. We demonstrate that cardinality sparsity can improve deep learning and tensor regression both statistically and computationally. Along the way, we generalize recent statistical theories in these fields. Most importantly, we show that cardinality sparsity has a strikingly powerful application beyond high-dimensional data analysis: it can significantly speed up matrix-matrix multiplications. For instance, we demonstrate that cardinality sparsity leads to algorithms for binary-matrix multiplication that outperform state-of-the-art algorithms by a substantial margin. Additionally, another crucial aspect of this sparsity is minimizing memory usage. By executing matrix multiplication in the compressed domain, we can significantly lower the amount of memory needed to store the input data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2302_08235 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Cardinality Sparsity: Applications in Matrix-Matrix Multiplications and Machine Learning Mohades, Ali Lederer, Johannes Statistics Theory High-dimensional data has become ubiquitous across the sciences but presents computational and statistical challenges. A common approach to addressing these challenges is through sparsity. In this paper, we introduce a new concept of sparsity, called cardinality sparsity. Broadly speaking, we define a tensor as sparse if it contains only a small number of unique values. We demonstrate that cardinality sparsity can improve deep learning and tensor regression both statistically and computationally. Along the way, we generalize recent statistical theories in these fields. Most importantly, we show that cardinality sparsity has a strikingly powerful application beyond high-dimensional data analysis: it can significantly speed up matrix-matrix multiplications. For instance, we demonstrate that cardinality sparsity leads to algorithms for binary-matrix multiplication that outperform state-of-the-art algorithms by a substantial margin. Additionally, another crucial aspect of this sparsity is minimizing memory usage. By executing matrix multiplication in the compressed domain, we can significantly lower the amount of memory needed to store the input data. |
| title | Cardinality Sparsity: Applications in Matrix-Matrix Multiplications and Machine Learning |
| topic | Statistics Theory |
| url | https://arxiv.org/abs/2302.08235 |