Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2411.18806 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915039318900736 |
|---|---|
| author | Xavier, Daniel Martin Chamoin, Ludovic Jerray, Jawher Fribourg, Laurent |
| author_facet | Xavier, Daniel Martin Chamoin, Ludovic Jerray, Jawher Fribourg, Laurent |
| contents | The early stopping strategy consists in stopping the training process of a neural network (NN) on a set $S$ of input data before training error is minimal. The advantage is that the NN then retains good generalization properties, i.e. it gives good predictions on data outside $S$, and a good estimate of the statistical error (``population loss'') is obtained. We give here an analytical estimation of the optimal stopping time involving basically the initial training error vector and the eigenvalues of the ``neural tangent kernel''. This yields an upper bound on the population loss which is well-suited to the underparameterized context (where the number of parameters is moderate compared with the number of data). Our method is illustrated on the example of an NN simulating the MPC control of a Van der Pol oscillator. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_18806 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | One-Step Early Stopping Strategy using Neural Tangent Kernel Theory and Rademacher Complexity Xavier, Daniel Martin Chamoin, Ludovic Jerray, Jawher Fribourg, Laurent Machine Learning Systems and Control The early stopping strategy consists in stopping the training process of a neural network (NN) on a set $S$ of input data before training error is minimal. The advantage is that the NN then retains good generalization properties, i.e. it gives good predictions on data outside $S$, and a good estimate of the statistical error (``population loss'') is obtained. We give here an analytical estimation of the optimal stopping time involving basically the initial training error vector and the eigenvalues of the ``neural tangent kernel''. This yields an upper bound on the population loss which is well-suited to the underparameterized context (where the number of parameters is moderate compared with the number of data). Our method is illustrated on the example of an NN simulating the MPC control of a Van der Pol oscillator. |
| title | One-Step Early Stopping Strategy using Neural Tangent Kernel Theory and Rademacher Complexity |
| topic | Machine Learning Systems and Control |
| url | https://arxiv.org/abs/2411.18806 |