Classification of Heavy-tailed Features in High Dimensions: a Superstatistical Approach
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911799855546368 |
|---|---|
| author | Adomaityte, Urte Sicuro, Gabriele Vivo, Pierpaolo |
| author_facet | Adomaityte, Urte Sicuro, Gabriele Vivo, Pierpaolo |
| contents | We characterise the learning of a mixture of two clouds of data points with generic centroids via empirical risk minimisation in the high dimensional regime, under the assumptions of generic convex loss and convex regularisation. Each cloud of data points is obtained via a double-stochastic process, where the sample is obtained from a Gaussian distribution whose variance is itself a random parameter sampled from a scalar distribution $\varrho$. As a result, our analysis covers a large family of data distributions, including the case of power-law-tailed distributions with no covariance, and allows us to test recent "Gaussian universality" claims. We study the generalisation performance of the obtained estimator, we analyse the role of regularisation, and we analytically characterise the separability transition. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2304_02912 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Classification of Heavy-tailed Features in High Dimensions: a Superstatistical Approach Adomaityte, Urte Sicuro, Gabriele Vivo, Pierpaolo Machine Learning Disordered Systems and Neural Networks Statistics Theory We characterise the learning of a mixture of two clouds of data points with generic centroids via empirical risk minimisation in the high dimensional regime, under the assumptions of generic convex loss and convex regularisation. Each cloud of data points is obtained via a double-stochastic process, where the sample is obtained from a Gaussian distribution whose variance is itself a random parameter sampled from a scalar distribution $\varrho$. As a result, our analysis covers a large family of data distributions, including the case of power-law-tailed distributions with no covariance, and allows us to test recent "Gaussian universality" claims. We study the generalisation performance of the obtained estimator, we analyse the role of regularisation, and we analytically characterise the separability transition. |
| title | Classification of Heavy-tailed Features in High Dimensions: a Superstatistical Approach |
| topic | Machine Learning Disordered Systems and Neural Networks Statistics Theory |
| url | https://arxiv.org/abs/2304.02912 |