GradTree: Learning Axis-Aligned Decision Trees with Gradient Descent
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910570332028928 |
|---|---|
| author | Marton, Sascha Lüdtke, Stefan Bartelt, Christian Stuckenschmidt, Heiner |
| author_facet | Marton, Sascha Lüdtke, Stefan Bartelt, Christian Stuckenschmidt, Heiner |
| contents | Decision Trees (DTs) are commonly used for many machine learning tasks due to their high degree of interpretability. However, learning a DT from data is a difficult optimization problem, as it is non-convex and non-differentiable. Therefore, common approaches learn DTs using a greedy growth algorithm that minimizes the impurity locally at each internal node. Unfortunately, this greedy procedure can lead to inaccurate trees. In this paper, we present a novel approach for learning hard, axis-aligned DTs with gradient descent. The proposed method uses backpropagation with a straight-through operator on a dense DT representation, to jointly optimize all tree parameters. Our approach outperforms existing methods on binary classification benchmarks and achieves competitive results for multi-class tasks. The method is available under: https://github.com/s-marton/GradTree |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_03515 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | GradTree: Learning Axis-Aligned Decision Trees with Gradient Descent Marton, Sascha Lüdtke, Stefan Bartelt, Christian Stuckenschmidt, Heiner Machine Learning Artificial Intelligence Decision Trees (DTs) are commonly used for many machine learning tasks due to their high degree of interpretability. However, learning a DT from data is a difficult optimization problem, as it is non-convex and non-differentiable. Therefore, common approaches learn DTs using a greedy growth algorithm that minimizes the impurity locally at each internal node. Unfortunately, this greedy procedure can lead to inaccurate trees. In this paper, we present a novel approach for learning hard, axis-aligned DTs with gradient descent. The proposed method uses backpropagation with a straight-through operator on a dense DT representation, to jointly optimize all tree parameters. Our approach outperforms existing methods on binary classification benchmarks and achieves competitive results for multi-class tasks. The method is available under: https://github.com/s-marton/GradTree |
| title | GradTree: Learning Axis-Aligned Decision Trees with Gradient Descent |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2305.03515 |