Divide, Specialize, and Route: A New Approach to Efficient Ensemble Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909660211052544 |
|---|---|
| author | Piwko, Jakub Ruciński, Jędrzej Płudowski, Dawid Zajko, Antoni Żak, Patryzja Zacharecki, Mateusz Kozak, Anna Woźnica, Katarzyna |
| author_facet | Piwko, Jakub Ruciński, Jędrzej Płudowski, Dawid Zajko, Antoni Żak, Patryzja Zacharecki, Mateusz Kozak, Anna Woźnica, Katarzyna |
| contents | Ensemble learning has proven effective in boosting predictive performance, but traditional methods such as bagging, boosting, and dynamic ensemble selection (DES) suffer from high computational cost and limited adaptability to heterogeneous data distributions. To address these limitations, we propose Hellsemble, a novel and interpretable ensemble framework for binary classification that leverages dataset complexity during both training and inference. Hellsemble incrementally partitions the dataset into circles of difficulty by iteratively passing misclassified instances from simpler models to subsequent ones, forming a committee of specialised base learners. Each model is trained on increasingly challenging subsets, while a separate router model learns to assign new instances to the most suitable base model based on inferred difficulty. Hellsemble achieves strong classification accuracy while maintaining computational efficiency and interpretability. Experimental results on OpenML-CC18 and Tabzilla benchmarks demonstrate that Hellsemble often outperforms classical ensemble methods. Our findings suggest that embracing instance-level difficulty offers a promising direction for constructing efficient and robust ensemble systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_20814 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Divide, Specialize, and Route: A New Approach to Efficient Ensemble Learning Piwko, Jakub Ruciński, Jędrzej Płudowski, Dawid Zajko, Antoni Żak, Patryzja Zacharecki, Mateusz Kozak, Anna Woźnica, Katarzyna Machine Learning Ensemble learning has proven effective in boosting predictive performance, but traditional methods such as bagging, boosting, and dynamic ensemble selection (DES) suffer from high computational cost and limited adaptability to heterogeneous data distributions. To address these limitations, we propose Hellsemble, a novel and interpretable ensemble framework for binary classification that leverages dataset complexity during both training and inference. Hellsemble incrementally partitions the dataset into circles of difficulty by iteratively passing misclassified instances from simpler models to subsequent ones, forming a committee of specialised base learners. Each model is trained on increasingly challenging subsets, while a separate router model learns to assign new instances to the most suitable base model based on inferred difficulty. Hellsemble achieves strong classification accuracy while maintaining computational efficiency and interpretability. Experimental results on OpenML-CC18 and Tabzilla benchmarks demonstrate that Hellsemble often outperforms classical ensemble methods. Our findings suggest that embracing instance-level difficulty offers a promising direction for constructing efficient and robust ensemble systems. |
| title | Divide, Specialize, and Route: A New Approach to Efficient Ensemble Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2506.20814 |