Invariant Kernels: Rank Stabilization and Generalization Across Dimensions
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916598284025856 |
|---|---|
| author | Díaz, Mateo Drusvyatskiy, Dmitriy Kendrick, Jack Thomas, Rekha R. |
| author_facet | Díaz, Mateo Drusvyatskiy, Dmitriy Kendrick, Jack Thomas, Rekha R. |
| contents | Symmetry arises often when learning from high dimensional data. For example, data sets consisting of point clouds, graphs, and unordered sets appear routinely in contemporary applications, and exhibit rich underlying symmetries. Understanding the benefits of symmetry on the statistical and numerical efficiency of learning algorithms is an active area of research. In this work, we show that symmetry has a pronounced impact on the rank of kernel matrices. Specifically, we compute the rank of a polynomial kernel of fixed degree that is invariant under various groups acting independently on its two arguments. In concrete circumstances, including the three aforementioned examples, symmetry dramatically decreases the rank making it independent of the data dimension. In such settings, we show that a simple regression procedure is minimax optimal for estimating an invariant polynomial from finitely many samples drawn across different dimensions. We complete the paper with numerical experiments that illustrate our findings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_01886 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Invariant Kernels: Rank Stabilization and Generalization Across Dimensions Díaz, Mateo Drusvyatskiy, Dmitriy Kendrick, Jack Thomas, Rekha R. Optimization and Control Representation Theory Machine Learning Symmetry arises often when learning from high dimensional data. For example, data sets consisting of point clouds, graphs, and unordered sets appear routinely in contemporary applications, and exhibit rich underlying symmetries. Understanding the benefits of symmetry on the statistical and numerical efficiency of learning algorithms is an active area of research. In this work, we show that symmetry has a pronounced impact on the rank of kernel matrices. Specifically, we compute the rank of a polynomial kernel of fixed degree that is invariant under various groups acting independently on its two arguments. In concrete circumstances, including the three aforementioned examples, symmetry dramatically decreases the rank making it independent of the data dimension. In such settings, we show that a simple regression procedure is minimax optimal for estimating an invariant polynomial from finitely many samples drawn across different dimensions. We complete the paper with numerical experiments that illustrate our findings. |
| title | Invariant Kernels: Rank Stabilization and Generalization Across Dimensions |
| topic | Optimization and Control Representation Theory Machine Learning |
| url | https://arxiv.org/abs/2502.01886 |