Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.08274 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915630395949056 |
|---|---|
| author | Pinheiro, João Manoel Herrera de Oliveira, Suzana Vilas Boas Silva, Thiago Henrique Segreto Saraiva, Pedro Antonio Rabelo de Souza, Enzo Ferreira Godoy, Ricardo V. Ambrosio, Leonardo André Becker, Marcelo |
| author_facet | Pinheiro, João Manoel Herrera de Oliveira, Suzana Vilas Boas Silva, Thiago Henrique Segreto Saraiva, Pedro Antonio Rabelo de Souza, Enzo Ferreira Godoy, Ricardo V. Ambrosio, Leonardo André Becker, Marcelo |
| contents | This research addresses the critical lack of comprehensive studies on feature scaling by systematically evaluating 12 scaling techniques - including several less common transformations - across 14 different Machine Learning algorithms and 16 datasets for classification and regression tasks. We meticulously analyzed impacts on predictive performance (using metrics such as accuracy, MAE, MSE, and $R^2$) and computational costs (training time, inference time, and memory usage). Key findings reveal that while ensemble methods (such as Random Forest and gradient boosting models like XGBoost, CatBoost and LightGBM) demonstrate robust performance largely independent of scaling, other widely used models such as Logistic Regression, SVMs, TabNet, and MLPs show significant performance variations highly dependent on the chosen scaler. This extensive empirical analysis, with all source code, experimental results, and model parameters made publicly available to ensure complete transparency and reproducibility, offers model-specific crucial guidance to practitioners on the need for an optimal selection of feature scaling techniques. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_08274 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Impact of Feature Scaling In Machine Learning: Effects on Regression and Classification Tasks Pinheiro, João Manoel Herrera de Oliveira, Suzana Vilas Boas Silva, Thiago Henrique Segreto Saraiva, Pedro Antonio Rabelo de Souza, Enzo Ferreira Godoy, Ricardo V. Ambrosio, Leonardo André Becker, Marcelo Machine Learning This research addresses the critical lack of comprehensive studies on feature scaling by systematically evaluating 12 scaling techniques - including several less common transformations - across 14 different Machine Learning algorithms and 16 datasets for classification and regression tasks. We meticulously analyzed impacts on predictive performance (using metrics such as accuracy, MAE, MSE, and $R^2$) and computational costs (training time, inference time, and memory usage). Key findings reveal that while ensemble methods (such as Random Forest and gradient boosting models like XGBoost, CatBoost and LightGBM) demonstrate robust performance largely independent of scaling, other widely used models such as Logistic Regression, SVMs, TabNet, and MLPs show significant performance variations highly dependent on the chosen scaler. This extensive empirical analysis, with all source code, experimental results, and model parameters made publicly available to ensure complete transparency and reproducibility, offers model-specific crucial guidance to practitioners on the need for an optimal selection of feature scaling techniques. |
| title | The Impact of Feature Scaling In Machine Learning: Effects on Regression and Classification Tasks |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2506.08274 |