Iteratively reweighted kernel machines efficiently learn sparse functions
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912624671719424 |
|---|---|
| author | Zhu, Libin Davis, Damek Drusvyatskiy, Dmitriy Fazel, Maryam |
| author_facet | Zhu, Libin Davis, Damek Drusvyatskiy, Dmitriy Fazel, Maryam |
| contents | The impressive practical performance of neural networks is often attributed to their ability to learn low-dimensional data representations and hierarchical structure directly from data. In this work, we argue that these two phenomena are not unique to neural networks, and can be elicited from classical kernel methods. Namely, we show that the derivative of the kernel predictor can detect the influential coordinates with low sample complexity. Moreover, by iteratively using the derivatives to reweight the data and retrain kernel machines, one is able to efficiently learn hierarchical polynomials with finite leap complexity. Numerical experiments illustrate the developed theory. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_08277 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Iteratively reweighted kernel machines efficiently learn sparse functions Zhu, Libin Davis, Damek Drusvyatskiy, Dmitriy Fazel, Maryam Machine Learning Optimization and Control Statistics Theory The impressive practical performance of neural networks is often attributed to their ability to learn low-dimensional data representations and hierarchical structure directly from data. In this work, we argue that these two phenomena are not unique to neural networks, and can be elicited from classical kernel methods. Namely, we show that the derivative of the kernel predictor can detect the influential coordinates with low sample complexity. Moreover, by iteratively using the derivatives to reweight the data and retrain kernel machines, one is able to efficiently learn hierarchical polynomials with finite leap complexity. Numerical experiments illustrate the developed theory. |
| title | Iteratively reweighted kernel machines efficiently learn sparse functions |
| topic | Machine Learning Optimization and Control Statistics Theory |
| url | https://arxiv.org/abs/2505.08277 |