Regularized least squares learning with heavy-tailed noise is minimax optimal
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914140925198336 |
|---|---|
| author | Mollenhauer, Mattes Mücke, Nicole Meunier, Dimitri Gretton, Arthur |
| author_facet | Mollenhauer, Mattes Mücke, Nicole Meunier, Dimitri Gretton, Arthur |
| contents | This paper examines the performance of ridge regression in reproducing kernel Hilbert spaces in the presence of noise that exhibits a finite number of higher moments. We establish excess risk bounds consisting of subgaussian and polynomial terms based on the well known integral operator framework. The dominant subgaussian component allows to achieve convergence rates that have previously only been derived under subexponential noise - a prevalent assumption in related work from the last two decades. These rates are optimal under standard eigenvalue decay conditions, demonstrating the asymptotic robustness of regularized least squares against heavy-tailed noise. Our derivations are based on a Fuk-Nagaev inequality for Hilbert-space valued random variables. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_14214 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Regularized least squares learning with heavy-tailed noise is minimax optimal Mollenhauer, Mattes Mücke, Nicole Meunier, Dimitri Gretton, Arthur Machine Learning Statistics Theory 62G08 (Primary) 62G35, 62J07 (Secondary) This paper examines the performance of ridge regression in reproducing kernel Hilbert spaces in the presence of noise that exhibits a finite number of higher moments. We establish excess risk bounds consisting of subgaussian and polynomial terms based on the well known integral operator framework. The dominant subgaussian component allows to achieve convergence rates that have previously only been derived under subexponential noise - a prevalent assumption in related work from the last two decades. These rates are optimal under standard eigenvalue decay conditions, demonstrating the asymptotic robustness of regularized least squares against heavy-tailed noise. Our derivations are based on a Fuk-Nagaev inequality for Hilbert-space valued random variables. |
| title | Regularized least squares learning with heavy-tailed noise is minimax optimal |
| topic | Machine Learning Statistics Theory 62G08 (Primary) 62G35, 62J07 (Secondary) |
| url | https://arxiv.org/abs/2505.14214 |