Geometry of Polynomial Neural Networks
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909413678252032 |
|---|---|
| author | Kubjas, Kaie Li, Jiayi Wiesmann, Maximilian |
| author_facet | Kubjas, Kaie Li, Jiayi Wiesmann, Maximilian |
| contents | We study the expressivity and learning process for polynomial neural networks (PNNs) with monomial activation functions. The weights of the network parametrize the neuromanifold. In this paper, we study certain neuromanifolds using tools from algebraic geometry: we give explicit descriptions as semialgebraic sets and characterize their Zariski closures, called neurovarieties. We study their dimension and associate an algebraic degree, the learning degree, to the neurovariety. The dimension serves as a geometric measure for the expressivity of the network, the learning degree is a measure for the complexity of training the network and provides upper bounds on the number of learnable functions. These theoretical results are accompanied with experiments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_00949 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Geometry of Polynomial Neural Networks Kubjas, Kaie Li, Jiayi Wiesmann, Maximilian Algebraic Geometry Machine Learning 68T07, 14P10, 14N07, 14M12 We study the expressivity and learning process for polynomial neural networks (PNNs) with monomial activation functions. The weights of the network parametrize the neuromanifold. In this paper, we study certain neuromanifolds using tools from algebraic geometry: we give explicit descriptions as semialgebraic sets and characterize their Zariski closures, called neurovarieties. We study their dimension and associate an algebraic degree, the learning degree, to the neurovariety. The dimension serves as a geometric measure for the expressivity of the network, the learning degree is a measure for the complexity of training the network and provides upper bounds on the number of learnable functions. These theoretical results are accompanied with experiments. |
| title | Geometry of Polynomial Neural Networks |
| topic | Algebraic Geometry Machine Learning 68T07, 14P10, 14N07, 14M12 |
| url | https://arxiv.org/abs/2402.00949 |