Convergence of machine learning methods for feedback control laws: averaged feedback learning scheme and data driven methods
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915306504454144 |
|---|---|
| author | Kunisch, Karl Vásquez-Varas, Donato |
| author_facet | Kunisch, Karl Vásquez-Varas, Donato |
| contents | This work addresses the synthesis of optimal feedback control laws via machine learning. In particular, the Averaged Feedback Learning Scheme (AFLS) and a data driven method are considered. Hypotheses for each method ensuring the convergence of the evaluation of the objective function of the underlying control problem at the obtained feedback-laws towards the optimal value function are provided. These hypotheses are connected to the regularity of the value function and the stability of the dynamics. In the case of AFLS these hypotheses only require Hölder continuity of the value function, whereas for the data driven method the value function must be at least $C^2$. It is demonstrated that these methods are connected via their optimality conditions. Additionally, numerical experiments are provided by applying both methods to a family control problems, parameterized by a positive real number which controls the regularity of the value function. For small parameters the value function is smooth and in contrast for large parameters it is non-differentiable, but semi-concave. The results of the experiments indicate that both methods have a similar performance for the case that the value function is smooth. On the other hand, if the value function is not differentiable, AFLS has a better performance which is consistent with the obtained convergence results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_18403 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Convergence of machine learning methods for feedback control laws: averaged feedback learning scheme and data driven methods Kunisch, Karl Vásquez-Varas, Donato Optimization and Control 49L12, 49J15, 49N35, 68Q32, 93B52 This work addresses the synthesis of optimal feedback control laws via machine learning. In particular, the Averaged Feedback Learning Scheme (AFLS) and a data driven method are considered. Hypotheses for each method ensuring the convergence of the evaluation of the objective function of the underlying control problem at the obtained feedback-laws towards the optimal value function are provided. These hypotheses are connected to the regularity of the value function and the stability of the dynamics. In the case of AFLS these hypotheses only require Hölder continuity of the value function, whereas for the data driven method the value function must be at least $C^2$. It is demonstrated that these methods are connected via their optimality conditions. Additionally, numerical experiments are provided by applying both methods to a family control problems, parameterized by a positive real number which controls the regularity of the value function. For small parameters the value function is smooth and in contrast for large parameters it is non-differentiable, but semi-concave. The results of the experiments indicate that both methods have a similar performance for the case that the value function is smooth. On the other hand, if the value function is not differentiable, AFLS has a better performance which is consistent with the obtained convergence results. |
| title | Convergence of machine learning methods for feedback control laws: averaged feedback learning scheme and data driven methods |
| topic | Optimization and Control 49L12, 49J15, 49N35, 68Q32, 93B52 |
| url | https://arxiv.org/abs/2407.18403 |