Algebraic Complexity and Neurovariety of Linear Convolutional Networks
Fuente:
arXiv
Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913215136399360 |
|---|---|
| author | Shahverdi, Vahid |
| author_facet | Shahverdi, Vahid |
| contents | In this paper, we study linear convolutional networks with one-dimensional filters and arbitrary strides. The neuromanifold of such a network is a semialgebraic set, represented by a space of polynomials admitting specific factorizations. Introducing a recursive algorithm, we generate polynomial equations whose common zero locus corresponds to the Zariski closure of the corresponding neuromanifold. Furthermore, we explore the algebraic complexity of training these networks employing tools from metric algebraic geometry. Our findings reveal that the number of all complex critical points in the optimization of such a network is equal to the generic Euclidean distance degree of a Segre variety. Notably, this count significantly surpasses the number of critical points encountered in the training of a fully connected linear network with the same number of parameters. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_16613 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Algebraic Complexity and Neurovariety of Linear Convolutional Networks Shahverdi, Vahid Algebraic Geometry Machine Learning 68T07, 14E99, 14J99, 14P10, 90C23 In this paper, we study linear convolutional networks with one-dimensional filters and arbitrary strides. The neuromanifold of such a network is a semialgebraic set, represented by a space of polynomials admitting specific factorizations. Introducing a recursive algorithm, we generate polynomial equations whose common zero locus corresponds to the Zariski closure of the corresponding neuromanifold. Furthermore, we explore the algebraic complexity of training these networks employing tools from metric algebraic geometry. Our findings reveal that the number of all complex critical points in the optimization of such a network is equal to the generic Euclidean distance degree of a Segre variety. Notably, this count significantly surpasses the number of critical points encountered in the training of a fully connected linear network with the same number of parameters. |
| title | Algebraic Complexity and Neurovariety of Linear Convolutional Networks |
| topic | Algebraic Geometry Machine Learning 68T07, 14E99, 14J99, 14P10, 90C23 |
| url | https://arxiv.org/abs/2401.16613 |