Data Compression using Rank-1 Lattices for Parameter Estimation in Machine Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909752473157632 |
|---|---|
| author | Gnewuch, Michael Harsha, Kumar Wnuk, Marcin |
| author_facet | Gnewuch, Michael Harsha, Kumar Wnuk, Marcin |
| contents | The mean squared error and regularized versions of it are standard loss functions in supervised machine learning. However, calculating these losses for large data sets can be computationally demanding. Modifying an approach of J. Dick and M. Feischl [Journal of Complexity 67 (2021)], we present algorithms to reduce extensive data sets to a smaller size using rank-1 lattices. Rank-1 lattices are quasi-Monte Carlo (QMC) point sets that are, if carefully chosen, well-distributed in a multidimensional unit cube. The compression strategy in the preprocessing step assigns every lattice point a pair of weights depending on the original data and responses, representing its relative importance. As a result, the compressed data makes iterative loss calculations in optimization steps much faster. We analyze the errors of our QMC data compression algorithms and the cost of the preprocessing step for functions whose Fourier coefficients decay sufficiently fast so that they lie in certain Wiener algebras or Korobov spaces. In particular, we prove that our approach can lead to arbitrary high convergence rates as long as the functions are sufficiently smooth. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_13453 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Data Compression using Rank-1 Lattices for Parameter Estimation in Machine Learning Gnewuch, Michael Harsha, Kumar Wnuk, Marcin Numerical Analysis Machine Learning 68Q32, 65D30, 42B05 (Primary) 11K38 (Secondary) F.2.1; G.1.2 The mean squared error and regularized versions of it are standard loss functions in supervised machine learning. However, calculating these losses for large data sets can be computationally demanding. Modifying an approach of J. Dick and M. Feischl [Journal of Complexity 67 (2021)], we present algorithms to reduce extensive data sets to a smaller size using rank-1 lattices. Rank-1 lattices are quasi-Monte Carlo (QMC) point sets that are, if carefully chosen, well-distributed in a multidimensional unit cube. The compression strategy in the preprocessing step assigns every lattice point a pair of weights depending on the original data and responses, representing its relative importance. As a result, the compressed data makes iterative loss calculations in optimization steps much faster. We analyze the errors of our QMC data compression algorithms and the cost of the preprocessing step for functions whose Fourier coefficients decay sufficiently fast so that they lie in certain Wiener algebras or Korobov spaces. In particular, we prove that our approach can lead to arbitrary high convergence rates as long as the functions are sufficiently smooth. |
| title | Data Compression using Rank-1 Lattices for Parameter Estimation in Machine Learning |
| topic | Numerical Analysis Machine Learning 68Q32, 65D30, 42B05 (Primary) 11K38 (Secondary) F.2.1; G.1.2 |
| url | https://arxiv.org/abs/2409.13453 |