Universal approximation property of ODENet and ResNet with a single activation function
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914983466500096 |
|---|---|
| author | Kimura, Masato Matsui, Kazunori Mizuno, Yosuke |
| author_facet | Kimura, Masato Matsui, Kazunori Mizuno, Yosuke |
| contents | We study a universal approximation property of ODENet and ResNet. The ODENet is a map from an initial value to the final value of an ODE system in a finite interval. It is considered a mathematical model of a ResNet-type deep learning system. We consider dynamical systems with vector fields given by a single composition of the activation function and an affine mapping, which is the most common choice of the ODENet or ResNet vector field in actual machine learning systems. We show that such an ODENet and ResNet with a restricted vector field can uniformly approximate ODENet with a general vector field. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_16709 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Universal approximation property of ODENet and ResNet with a single activation function Kimura, Masato Matsui, Kazunori Mizuno, Yosuke Machine Learning Artificial Intelligence Numerical Analysis 37M15, 41A46, 41A63, 65L12, 68T07, 65P99 We study a universal approximation property of ODENet and ResNet. The ODENet is a map from an initial value to the final value of an ODE system in a finite interval. It is considered a mathematical model of a ResNet-type deep learning system. We consider dynamical systems with vector fields given by a single composition of the activation function and an affine mapping, which is the most common choice of the ODENet or ResNet vector field in actual machine learning systems. We show that such an ODENet and ResNet with a restricted vector field can uniformly approximate ODENet with a general vector field. |
| title | Universal approximation property of ODENet and ResNet with a single activation function |
| topic | Machine Learning Artificial Intelligence Numerical Analysis 37M15, 41A46, 41A63, 65L12, 68T07, 65P99 |
| url | https://arxiv.org/abs/2410.16709 |