Consistency of Random Forest Type Algorithms under a Probabilistic Impurity Decrease Condition
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911780958109696 |
|---|---|
| author | Blum, Ricardo Hiabu, Munir Mammen, Enno Meyer, Joseph T. |
| author_facet | Blum, Ricardo Hiabu, Munir Mammen, Enno Meyer, Joseph T. |
| contents | This paper derives a unifying theorem establishing consistency results for a broad class of tree-based algorithms. It improves current results in two aspects. First of all, it can be applied to algorithms that vary from traditional Random Forests due to additional randomness for choosing splits, extending split options, allowing partitions into more than two cells in a single iteration step, and combinations of those. In particular, we prove consistency for Extremely Randomized Trees, Interaction Forests and Oblique Regression Trees using our general theorem. Secondly, it can be used to demonstrate consistency for a larger function class compared to previous results on Random Forests if one allows for additional random splits. Our results are based on the extension of the recently introduced notion of sufficient impurity decrease to a probabilistic sufficient impurity decrease condition. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_01460 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Consistency of Random Forest Type Algorithms under a Probabilistic Impurity Decrease Condition Blum, Ricardo Hiabu, Munir Mammen, Enno Meyer, Joseph T. Statistics Theory This paper derives a unifying theorem establishing consistency results for a broad class of tree-based algorithms. It improves current results in two aspects. First of all, it can be applied to algorithms that vary from traditional Random Forests due to additional randomness for choosing splits, extending split options, allowing partitions into more than two cells in a single iteration step, and combinations of those. In particular, we prove consistency for Extremely Randomized Trees, Interaction Forests and Oblique Regression Trees using our general theorem. Secondly, it can be used to demonstrate consistency for a larger function class compared to previous results on Random Forests if one allows for additional random splits. Our results are based on the extension of the recently introduced notion of sufficient impurity decrease to a probabilistic sufficient impurity decrease condition. |
| title | Consistency of Random Forest Type Algorithms under a Probabilistic Impurity Decrease Condition |
| topic | Statistics Theory |
| url | https://arxiv.org/abs/2309.01460 |