Approximation and interpolation of deep neural networks
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911851921539072 |
|---|---|
| author | Constantinescu, Vlad-Raul Popescu, Ionel |
| author_facet | Constantinescu, Vlad-Raul Popescu, Ionel |
| contents | In this paper, we prove that in the overparametrized regime, deep neural network provide universal approximations and can interpolate any data set, as long as the activation function is locally in $L^1(\RR)$ and not an affine function.
Additionally, if the activation function is smooth and such an interpolation networks exists, then the set of parameters which interpolate forms a manifold. Furthermore, we give a characterization of the Hessian of the loss function evaluated at the interpolation points.
In the last section, we provide a practical probabilistic method of finding such a point under general conditions on the activation function. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2304_10552 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Approximation and interpolation of deep neural networks Constantinescu, Vlad-Raul Popescu, Ionel Machine Learning Optimization and Control Probability In this paper, we prove that in the overparametrized regime, deep neural network provide universal approximations and can interpolate any data set, as long as the activation function is locally in $L^1(\RR)$ and not an affine function. Additionally, if the activation function is smooth and such an interpolation networks exists, then the set of parameters which interpolate forms a manifold. Furthermore, we give a characterization of the Hessian of the loss function evaluated at the interpolation points. In the last section, we provide a practical probabilistic method of finding such a point under general conditions on the activation function. |
| title | Approximation and interpolation of deep neural networks |
| topic | Machine Learning Optimization and Control Probability |
| url | https://arxiv.org/abs/2304.10552 |