Sparse Projection Oblique Randomer Forests
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2015
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866907957712650240 |
|---|---|
| author | Tomita, Tyler M. Browne, James Shen, Cencheng Chung, Jaewon Patsolic, Jesse L. Falk, Benjamin Yim, Jason Priebe, Carey E. Burns, Randal Maggioni, Mauro Vogelstein, Joshua T. |
| author_facet | Tomita, Tyler M. Browne, James Shen, Cencheng Chung, Jaewon Patsolic, Jesse L. Falk, Benjamin Yim, Jason Priebe, Carey E. Burns, Randal Maggioni, Mauro Vogelstein, Joshua T. |
| contents | Decision forests, including Random Forests and Gradient Boosting Trees, have recently demonstrated state-of-the-art performance in a variety of machine learning settings. Decision forests are typically ensembles of axis-aligned decision trees; that is, trees that split only along feature dimensions. In contrast, many recent extensions to decision forests are based on axis-oblique splits. Unfortunately, these extensions forfeit one or more of the favorable properties of decision forests based on axis-aligned splits, such as robustness to many noise dimensions, interpretability, or computational efficiency. We introduce yet another decision forest, called "Sparse Projection Oblique Randomer Forests" (SPORF). SPORF uses very sparse random projections, i.e., linear combinations of a small subset of features. SPORF significantly improves accuracy over existing state-of-the-art algorithms on a standard benchmark suite for classification with >100 problems of varying dimension, sample size, and number of classes. To illustrate how SPORF addresses the limitations of both axis-aligned and existing oblique decision forest methods, we conduct extensive simulated experiments. SPORF typically yields improved performance over existing decision forests, while mitigating computational efficiency and scalability and maintaining interpretability. SPORF can easily be incorporated into other ensemble methods such as boosting to obtain potentially similar gains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_1506_03410 |
| institution | arXiv |
| publishDate | 2015 |
| record_format | arxiv |
| spellingShingle | Sparse Projection Oblique Randomer Forests Tomita, Tyler M. Browne, James Shen, Cencheng Chung, Jaewon Patsolic, Jesse L. Falk, Benjamin Yim, Jason Priebe, Carey E. Burns, Randal Maggioni, Mauro Vogelstein, Joshua T. Machine Learning 68T10 I.5.2 Decision forests, including Random Forests and Gradient Boosting Trees, have recently demonstrated state-of-the-art performance in a variety of machine learning settings. Decision forests are typically ensembles of axis-aligned decision trees; that is, trees that split only along feature dimensions. In contrast, many recent extensions to decision forests are based on axis-oblique splits. Unfortunately, these extensions forfeit one or more of the favorable properties of decision forests based on axis-aligned splits, such as robustness to many noise dimensions, interpretability, or computational efficiency. We introduce yet another decision forest, called "Sparse Projection Oblique Randomer Forests" (SPORF). SPORF uses very sparse random projections, i.e., linear combinations of a small subset of features. SPORF significantly improves accuracy over existing state-of-the-art algorithms on a standard benchmark suite for classification with >100 problems of varying dimension, sample size, and number of classes. To illustrate how SPORF addresses the limitations of both axis-aligned and existing oblique decision forest methods, we conduct extensive simulated experiments. SPORF typically yields improved performance over existing decision forests, while mitigating computational efficiency and scalability and maintaining interpretability. SPORF can easily be incorporated into other ensemble methods such as boosting to obtain potentially similar gains. |
| title | Sparse Projection Oblique Randomer Forests |
| topic | Machine Learning 68T10 I.5.2 |
| url | https://arxiv.org/abs/1506.03410 |