An axiomatized PDE model of deep neural networks
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929285590155264 |
|---|---|
| author | Wang, Tangjun Tao, Wenqi Bao, Chenglong Shi, Zuoqiang |
| author_facet | Wang, Tangjun Tao, Wenqi Bao, Chenglong Shi, Zuoqiang |
| contents | Inspired by the relation between deep neural network (DNN) and partial differential equations (PDEs), we study the general form of the PDE models of deep neural networks. To achieve this goal, we formulate DNN as an evolution operator from a simple base model. Based on several reasonable assumptions, we prove that the evolution operator is actually determined by convection-diffusion equation. This convection-diffusion equation model gives mathematical explanation for several effective networks. Moreover, we show that the convection-diffusion model improves the robustness and reduces the Rademacher complexity. Based on the convection-diffusion equation, we design a new training method for ResNets. Experiments validate the performance of the proposed method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2307_12333 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | An axiomatized PDE model of deep neural networks Wang, Tangjun Tao, Wenqi Bao, Chenglong Shi, Zuoqiang Machine Learning Inspired by the relation between deep neural network (DNN) and partial differential equations (PDEs), we study the general form of the PDE models of deep neural networks. To achieve this goal, we formulate DNN as an evolution operator from a simple base model. Based on several reasonable assumptions, we prove that the evolution operator is actually determined by convection-diffusion equation. This convection-diffusion equation model gives mathematical explanation for several effective networks. Moreover, we show that the convection-diffusion model improves the robustness and reduces the Rademacher complexity. Based on the convection-diffusion equation, we design a new training method for ResNets. Experiments validate the performance of the proposed method. |
| title | An axiomatized PDE model of deep neural networks |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2307.12333 |