Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.14158 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914133110161408 |
|---|---|
| author | Ruan, Feng Liu, Keli Jordan, Michael |
| author_facet | Ruan, Feng Liu, Keli Jordan, Michael |
| contents | We study a compositional variant of kernel ridge regression in which the predictor is applied to a coordinate-wise reweighting of the inputs. Formulated as a variational problem, this model provides a simple testbed for feature learning in compositional architectures. From the perspective of variable selection, we show how relevant variables are recovered while noise variables are eliminated. We establish guarantees showing that both global minimizers and stationary points discard noise coordinates when the noise variables are Gaussian distributed. A central finding is that $\ell_1$-type kernels, such as the Laplace kernel, succeed in recovering features contributing to nonlinear effects at stationary points, whereas Gaussian kernels recover only linear ones. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_14158 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Compositional Kernel Model for Feature Learning Ruan, Feng Liu, Keli Jordan, Michael Machine Learning Optimization and Control We study a compositional variant of kernel ridge regression in which the predictor is applied to a coordinate-wise reweighting of the inputs. Formulated as a variational problem, this model provides a simple testbed for feature learning in compositional architectures. From the perspective of variable selection, we show how relevant variables are recovered while noise variables are eliminated. We establish guarantees showing that both global minimizers and stationary points discard noise coordinates when the noise variables are Gaussian distributed. A central finding is that $\ell_1$-type kernels, such as the Laplace kernel, succeed in recovering features contributing to nonlinear effects at stationary points, whereas Gaussian kernels recover only linear ones. |
| title | A Compositional Kernel Model for Feature Learning |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2509.14158 |