High-Dimensional Analysis of Bootstrap Ensemble Classifiers
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916007717634048 |
|---|---|
| author | Tiomoko, Malik Cherkaoui, Hamza Seddik, Mohamed El Amine Louart, Cosme Schnoor, Ekkehard Kegl, Balazs |
| author_facet | Tiomoko, Malik Cherkaoui, Hamza Seddik, Mohamed El Amine Louart, Cosme Schnoor, Ekkehard Kegl, Balazs |
| contents | Bootstrap methods have long been the cornerstone of ensemble learning in machine learning. This paper presents a theoretical analysis of bootstrap techniques applied to the Least Square Support Vector Machine (LSSVM) ensemble in the context of large and growing sample sizes and feature dimensionalities. Using tools from Random Matrix Theory, we investigate the performance of this classifier that aggregates decision functions from multiple weak classifiers, each trained on different subsets of the data. We provide insights into the use of bootstrap methods in high-dimensional settings, enhancing our understanding of their impact. Based on these findings, we propose strategies to select the number of subsets and the regularization parameter that maximize the performance of the LSSVM. Empirical experiments on synthetic and real-world datasets validate our theoretical results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_14587 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | High-Dimensional Analysis of Bootstrap Ensemble Classifiers Tiomoko, Malik Cherkaoui, Hamza Seddik, Mohamed El Amine Louart, Cosme Schnoor, Ekkehard Kegl, Balazs Machine Learning Bootstrap methods have long been the cornerstone of ensemble learning in machine learning. This paper presents a theoretical analysis of bootstrap techniques applied to the Least Square Support Vector Machine (LSSVM) ensemble in the context of large and growing sample sizes and feature dimensionalities. Using tools from Random Matrix Theory, we investigate the performance of this classifier that aggregates decision functions from multiple weak classifiers, each trained on different subsets of the data. We provide insights into the use of bootstrap methods in high-dimensional settings, enhancing our understanding of their impact. Based on these findings, we propose strategies to select the number of subsets and the regularization parameter that maximize the performance of the LSSVM. Empirical experiments on synthetic and real-world datasets validate our theoretical results. |
| title | High-Dimensional Analysis of Bootstrap Ensemble Classifiers |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2505.14587 |