EarlyStopping: Implicit Regularization for Iterative Learning Procedures in Python
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916658856067072 |
|---|---|
| author | Ziebell, Eric Miftachov, Ratmir Stankewitz, Bernhard Hucker, Laura |
| author_facet | Ziebell, Eric Miftachov, Ratmir Stankewitz, Bernhard Hucker, Laura |
| contents | Iterative learning procedures are ubiquitous in machine learning and modern statistics.
Regularision is typically required to prevent inflating the expected loss of a procedure in
later iterations via the propagation of noise inherent in the data.
Significant emphasis has been placed on achieving this regularisation implicitly by stopping
procedures early.
The EarlyStopping-package provides a toolbox of (in-sample) sequential early stopping rules for
several well-known iterative estimation procedures, such as truncated SVD, Landweber (gradient
descent), conjugate gradient descent, L2-boosting and regression trees.
One of the central features of the package is that the algorithms allow the specification of the
true data-generating process and keep track of relevant theoretical quantities.
In this paper, we detail the principles governing the implementation of the EarlyStopping-package and provide
a survey of recent foundational advances in the theoretical literature.
We demonstrate how to use the EarlyStopping-package to explore core features of implicit regularisation
and replicate results from the literature. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_16753 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | EarlyStopping: Implicit Regularization for Iterative Learning Procedures in Python Ziebell, Eric Miftachov, Ratmir Stankewitz, Bernhard Hucker, Laura Machine Learning Mathematical Software Iterative learning procedures are ubiquitous in machine learning and modern statistics. Regularision is typically required to prevent inflating the expected loss of a procedure in later iterations via the propagation of noise inherent in the data. Significant emphasis has been placed on achieving this regularisation implicitly by stopping procedures early. The EarlyStopping-package provides a toolbox of (in-sample) sequential early stopping rules for several well-known iterative estimation procedures, such as truncated SVD, Landweber (gradient descent), conjugate gradient descent, L2-boosting and regression trees. One of the central features of the package is that the algorithms allow the specification of the true data-generating process and keep track of relevant theoretical quantities. In this paper, we detail the principles governing the implementation of the EarlyStopping-package and provide a survey of recent foundational advances in the theoretical literature. We demonstrate how to use the EarlyStopping-package to explore core features of implicit regularisation and replicate results from the literature. |
| title | EarlyStopping: Implicit Regularization for Iterative Learning Procedures in Python |
| topic | Machine Learning Mathematical Software |
| url | https://arxiv.org/abs/2503.16753 |