Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2404.05185 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909652020625408 |
|---|---|
| author | Liao, Huafu Mészáros, Alpár R. Mou, Chenchen Zhou, Chao |
| author_facet | Liao, Huafu Mészáros, Alpár R. Mou, Chenchen Zhou, Chao |
| contents | This paper deals with a class of neural SDEs and studies the limiting behavior of the associated sampled optimal control problems as the sample size grows to infinity. The neural SDEs with $N$ samples can be linked to the $N$-particle systems with centralized control. We analyze the Hamilton-Jacobi-Bellman equation corresponding to the $N$-particle system and establish regularity results which are uniform in $N$. The uniform regularity estimates are obtained by the stochastic maximum principle and the analysis of a backward stochastic Riccati equation. Using these uniform regularity results, we show the convergence of the minima of the objective functionals and optimal parameters of the neural SDEs as the sample size $N$ tends to infinity. The limiting objects can be identified with suitable functions defined on the Wasserstein space of Borel probability measures. Furthermore, quantitative convergence rates are also obtained. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_05185 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Convergence analysis of controlled particle systems arising in deep learning: from finite to infinite sample size Liao, Huafu Mészáros, Alpár R. Mou, Chenchen Zhou, Chao Optimization and Control Machine Learning Probability 49N80, 65C35, 49L12, 62M45 This paper deals with a class of neural SDEs and studies the limiting behavior of the associated sampled optimal control problems as the sample size grows to infinity. The neural SDEs with $N$ samples can be linked to the $N$-particle systems with centralized control. We analyze the Hamilton-Jacobi-Bellman equation corresponding to the $N$-particle system and establish regularity results which are uniform in $N$. The uniform regularity estimates are obtained by the stochastic maximum principle and the analysis of a backward stochastic Riccati equation. Using these uniform regularity results, we show the convergence of the minima of the objective functionals and optimal parameters of the neural SDEs as the sample size $N$ tends to infinity. The limiting objects can be identified with suitable functions defined on the Wasserstein space of Borel probability measures. Furthermore, quantitative convergence rates are also obtained. |
| title | Convergence analysis of controlled particle systems arising in deep learning: from finite to infinite sample size |
| topic | Optimization and Control Machine Learning Probability 49N80, 65C35, 49L12, 62M45 |
| url | https://arxiv.org/abs/2404.05185 |