Deep Learning without Global Optimization by Random Fourier Neural Networks
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916644773691392 |
|---|---|
| author | Davis, Owen Geraci, Gianluca Motamed, Mohammad |
| author_facet | Davis, Owen Geraci, Gianluca Motamed, Mohammad |
| contents | We introduce a new training algorithm for deep neural networks that utilize random complex exponential activation functions. Our approach employs a Markov Chain Monte Carlo sampling procedure to iteratively train network layers, avoiding global and gradient-based optimization while maintaining error control. It consistently attains the theoretical approximation rate for residual networks with complex exponential activation functions, determined by network complexity. Additionally, it enables efficient learning of multiscale and high-frequency features, producing interpretable parameter distributions. Despite using sinusoidal basis functions, we do not observe Gibbs phenomena in approximating discontinuous target functions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_11894 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Deep Learning without Global Optimization by Random Fourier Neural Networks Davis, Owen Geraci, Gianluca Motamed, Mohammad Machine Learning Numerical Analysis 65T40, 90C15, 65C05, 65C40, 60J22, 68T07 We introduce a new training algorithm for deep neural networks that utilize random complex exponential activation functions. Our approach employs a Markov Chain Monte Carlo sampling procedure to iteratively train network layers, avoiding global and gradient-based optimization while maintaining error control. It consistently attains the theoretical approximation rate for residual networks with complex exponential activation functions, determined by network complexity. Additionally, it enables efficient learning of multiscale and high-frequency features, producing interpretable parameter distributions. Despite using sinusoidal basis functions, we do not observe Gibbs phenomena in approximating discontinuous target functions. |
| title | Deep Learning without Global Optimization by Random Fourier Neural Networks |
| topic | Machine Learning Numerical Analysis 65T40, 90C15, 65C05, 65C40, 60J22, 68T07 |
| url | https://arxiv.org/abs/2407.11894 |