Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.12185 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909789185900544 |
|---|---|
| author | Arratia, Argimiro Cabaña, Alejandra Mordecki, Ernesto Rovira-Parra, Gerard |
| author_facet | Arratia, Argimiro Cabaña, Alejandra Mordecki, Ernesto Rovira-Parra, Gerard |
| contents | Model selection in non-linear models often prioritizes performance metrics over statistical tests, limiting the ability to account for sampling variability. We propose the use of a statistical test to assess the equality of variances in forecasting errors. The test builds upon the classic Morgan-Pitman approach, incorporating enhancements to ensure robustness against data with heavy-tailed distributions or outliers with high variance, plus a strategy to make residuals from machine learning models statistically independent. Through a series of simulations and real-world data applications, we demonstrate the test's effectiveness and practical utility, offering a reliable tool for model evaluation and selection in diverse contexts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_12185 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Morgan-Pitman Test of Equality of Variances and its Application to Machine Learning Model Evaluation and Selection Arratia, Argimiro Cabaña, Alejandra Mordecki, Ernesto Rovira-Parra, Gerard Machine Learning Statistics Theory Model selection in non-linear models often prioritizes performance metrics over statistical tests, limiting the ability to account for sampling variability. We propose the use of a statistical test to assess the equality of variances in forecasting errors. The test builds upon the classic Morgan-Pitman approach, incorporating enhancements to ensure robustness against data with heavy-tailed distributions or outliers with high variance, plus a strategy to make residuals from machine learning models statistically independent. Through a series of simulations and real-world data applications, we demonstrate the test's effectiveness and practical utility, offering a reliable tool for model evaluation and selection in diverse contexts. |
| title | The Morgan-Pitman Test of Equality of Variances and its Application to Machine Learning Model Evaluation and Selection |
| topic | Machine Learning Statistics Theory |
| url | https://arxiv.org/abs/2509.12185 |