Enhance Learning Efficiency of Oblique Decision Tree via Feature Concatenation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909472014729216 |
|---|---|
| author | Lyu, Shen-Huan He, Yi-Xiao Wang, Yanyan Qu, Zhihao Tang, Bin Ye, Baoliu |
| author_facet | Lyu, Shen-Huan He, Yi-Xiao Wang, Yanyan Qu, Zhihao Tang, Bin Ye, Baoliu |
| contents | Oblique Decision Tree (ODT) separates the feature space by linear projections, as opposed to the conventional Decision Tree (DT) that forces axis-parallel splits. ODT has been proven to have a stronger representation ability than DT, as it provides a way to create shallower tree structures while still approximating complex decision boundaries. However, its learning efficiency is still insufficient, since the linear projections cannot be transmitted to the child nodes, resulting in a waste of model parameters. In this work, we propose an enhanced ODT method with Feature Concatenation (\texttt{FC-ODT}), which enables in-model feature transformation to transmit the projections along the decision paths. Theoretically, we prove that our method enjoys a faster consistency rate w.r.t. the tree depth, indicating that our method possesses a significant advantage in generalization performance, especially for shallow trees. Experiments show that \texttt{FC-ODT} can outperform the other state-of-the-art decision trees with a limited tree depth. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_00465 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Enhance Learning Efficiency of Oblique Decision Tree via Feature Concatenation Lyu, Shen-Huan He, Yi-Xiao Wang, Yanyan Qu, Zhihao Tang, Bin Ye, Baoliu Machine Learning Artificial Intelligence Oblique Decision Tree (ODT) separates the feature space by linear projections, as opposed to the conventional Decision Tree (DT) that forces axis-parallel splits. ODT has been proven to have a stronger representation ability than DT, as it provides a way to create shallower tree structures while still approximating complex decision boundaries. However, its learning efficiency is still insufficient, since the linear projections cannot be transmitted to the child nodes, resulting in a waste of model parameters. In this work, we propose an enhanced ODT method with Feature Concatenation (\texttt{FC-ODT}), which enables in-model feature transformation to transmit the projections along the decision paths. Theoretically, we prove that our method enjoys a faster consistency rate w.r.t. the tree depth, indicating that our method possesses a significant advantage in generalization performance, especially for shallow trees. Experiments show that \texttt{FC-ODT} can outperform the other state-of-the-art decision trees with a limited tree depth. |
| title | Enhance Learning Efficiency of Oblique Decision Tree via Feature Concatenation |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2502.00465 |