Early Stopping Based on Repeated Significance
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916344064114688 |
|---|---|
| author | Bax, Eric Sarkar, Arundhyoti Shtoff, Alex |
| author_facet | Bax, Eric Sarkar, Arundhyoti Shtoff, Alex |
| contents | For a bucket test with a single criterion for success and a fixed number of samples or testing period, requiring a $p$-value less than a specified value of $α$ for the success criterion produces statistical confidence at level $1 - α$. For multiple criteria, a Bonferroni correction that partitions $α$ among the criteria produces statistical confidence, at the cost of requiring lower $p$-values for each criterion. The same concept can be applied to decisions about early stopping, but that can lead to strict requirements for $p$-values. We show how to address that challenge by requiring criteria to be successful at multiple decision points. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_00908 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Early Stopping Based on Repeated Significance Bax, Eric Sarkar, Arundhyoti Shtoff, Alex Methodology Machine Learning For a bucket test with a single criterion for success and a fixed number of samples or testing period, requiring a $p$-value less than a specified value of $α$ for the success criterion produces statistical confidence at level $1 - α$. For multiple criteria, a Bonferroni correction that partitions $α$ among the criteria produces statistical confidence, at the cost of requiring lower $p$-values for each criterion. The same concept can be applied to decisions about early stopping, but that can lead to strict requirements for $p$-values. We show how to address that challenge by requiring criteria to be successful at multiple decision points. |
| title | Early Stopping Based on Repeated Significance |
| topic | Methodology Machine Learning |
| url | https://arxiv.org/abs/2408.00908 |