Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.05108 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916593022271488 |
|---|---|
| author | Lyu, Shen-Huan Wu, Jin-Hui Zheng, Qin-Cheng Ye, Baoliu |
| author_facet | Lyu, Shen-Huan Wu, Jin-Hui Zheng, Qin-Cheng Ye, Baoliu |
| contents | Random forests are classical ensemble algorithms that construct multiple randomized decision trees and aggregate their predictions using naive averaging. \citet{zhou2019deep} further propose a deep forest algorithm with multi-layer forests, which outperforms random forests in various tasks. The performance of deep forests is related to three hyperparameters in practice: depth, width, and tree size, but little has been known about its theoretical explanation. This work provides the first upper and lower bounds on the approximation complexity of deep forests concerning the three hyperparameters. Our results confirm the distinctive role of depth, which can exponentially enhance the expressiveness of deep forests compared with width and tree size. Experiments confirm the theoretical findings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_05108 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Role of Depth, Width, and Tree Size in Expressiveness of Deep Forest Lyu, Shen-Huan Wu, Jin-Hui Zheng, Qin-Cheng Ye, Baoliu Machine Learning Random forests are classical ensemble algorithms that construct multiple randomized decision trees and aggregate their predictions using naive averaging. \citet{zhou2019deep} further propose a deep forest algorithm with multi-layer forests, which outperforms random forests in various tasks. The performance of deep forests is related to three hyperparameters in practice: depth, width, and tree size, but little has been known about its theoretical explanation. This work provides the first upper and lower bounds on the approximation complexity of deep forests concerning the three hyperparameters. Our results confirm the distinctive role of depth, which can exponentially enhance the expressiveness of deep forests compared with width and tree size. Experiments confirm the theoretical findings. |
| title | The Role of Depth, Width, and Tree Size in Expressiveness of Deep Forest |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2407.05108 |