Simplified derivations for high-dimensional convex learning problems
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909874063933440 |
|---|---|
| author | Clark, David G. Sompolinsky, Haim |
| author_facet | Clark, David G. Sompolinsky, Haim |
| contents | Statistical-physics calculations in machine learning and theoretical neuroscience often involve lengthy derivations that obscure physical interpretation. Here, we give concise, non-replica derivations of several key results and highlight their underlying similarities. In particular, using a cavity approach, we analyze three high-dimensional learning problems: perceptron classification of points, perceptron classification of manifolds, and kernel ridge regression. These problems share a common structure--a bipartite system of interacting feature and datum variables--enabling a unified analysis. Furthermore, for perceptron-capacity problems, we identify a symmetry that allows derivation of correct capacities through a naive method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_01110 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Simplified derivations for high-dimensional convex learning problems Clark, David G. Sompolinsky, Haim Disordered Systems and Neural Networks Neural and Evolutionary Computing Neurons and Cognition Statistical-physics calculations in machine learning and theoretical neuroscience often involve lengthy derivations that obscure physical interpretation. Here, we give concise, non-replica derivations of several key results and highlight their underlying similarities. In particular, using a cavity approach, we analyze three high-dimensional learning problems: perceptron classification of points, perceptron classification of manifolds, and kernel ridge regression. These problems share a common structure--a bipartite system of interacting feature and datum variables--enabling a unified analysis. Furthermore, for perceptron-capacity problems, we identify a symmetry that allows derivation of correct capacities through a naive method. |
| title | Simplified derivations for high-dimensional convex learning problems |
| topic | Disordered Systems and Neural Networks Neural and Evolutionary Computing Neurons and Cognition |
| url | https://arxiv.org/abs/2412.01110 |