Learning Polynomial Activation Functions for Deep Neural Networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916990101225472 |
|---|---|
| author | Zhang, Linghao Nie, Jiawang Tang, Tingting |
| author_facet | Zhang, Linghao Nie, Jiawang Tang, Tingting |
| contents | Activation functions are crucial for deep neural networks. This novel work frames the problem of training neural network with learnable polynomial activation functions as a polynomial optimization problem, which is solvable by the Moment-SOS hierarchy. This work represents a fundamental departure from the conventional paradigm of training deep neural networks, which relies on local optimization methods like backpropagation and gradient descent. Numerical experiments are presented to demonstrate the accuracy and robustness of optimum parameter recovery in presence of noises. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_03682 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Learning Polynomial Activation Functions for Deep Neural Networks Zhang, Linghao Nie, Jiawang Tang, Tingting Optimization and Control Activation functions are crucial for deep neural networks. This novel work frames the problem of training neural network with learnable polynomial activation functions as a polynomial optimization problem, which is solvable by the Moment-SOS hierarchy. This work represents a fundamental departure from the conventional paradigm of training deep neural networks, which relies on local optimization methods like backpropagation and gradient descent. Numerical experiments are presented to demonstrate the accuracy and robustness of optimum parameter recovery in presence of noises. |
| title | Learning Polynomial Activation Functions for Deep Neural Networks |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2510.03682 |