Decision Trees That Remember: Gradient-Based Learning of Recurrent Decision Trees with Memory

Fuente: arXiv
Guardado en:
Detalles Bibliográficos
Autores principales: Marton, Sascha, Schneider, Moritz
Formato: Preprint
Publicado: 2025
Materias:
Acceso en línea:
Etiquetas: Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
_version_ 1866915141534089216
author Marton, Sascha
Schneider, Moritz
author_facet Marton, Sascha
Schneider, Moritz
contents Neural architectures such as Recurrent Neural Networks (RNNs), Transformers, and State-Space Models have shown great success in handling sequential data by learning temporal dependencies. Decision Trees (DTs), on the other hand, remain a widely used class of models for structured tabular data but are typically not designed to capture sequential patterns directly. Instead, DT-based approaches for time-series data often rely on feature engineering, such as manually incorporating lag features, which can be suboptimal for capturing complex temporal dependencies. To address this limitation, we introduce ReMeDe Trees, a novel recurrent DT architecture that integrates an internal memory mechanism, similar to RNNs, to learn long-term dependencies in sequential data. Our model learns hard, axis-aligned decision rules for both output generation and state updates, optimizing them efficiently via gradient descent. We provide a proof-of-concept study on synthetic benchmarks to demonstrate the effectiveness of our approach.
format Preprint
id arxiv_https___arxiv_org_abs_2502_04052
institution arXiv
publishDate 2025
record_format arxiv
spellingShingle Decision Trees That Remember: Gradient-Based Learning of Recurrent Decision Trees with Memory
Marton, Sascha
Schneider, Moritz
Machine Learning
Neural architectures such as Recurrent Neural Networks (RNNs), Transformers, and State-Space Models have shown great success in handling sequential data by learning temporal dependencies. Decision Trees (DTs), on the other hand, remain a widely used class of models for structured tabular data but are typically not designed to capture sequential patterns directly. Instead, DT-based approaches for time-series data often rely on feature engineering, such as manually incorporating lag features, which can be suboptimal for capturing complex temporal dependencies. To address this limitation, we introduce ReMeDe Trees, a novel recurrent DT architecture that integrates an internal memory mechanism, similar to RNNs, to learn long-term dependencies in sequential data. Our model learns hard, axis-aligned decision rules for both output generation and state updates, optimizing them efficiently via gradient descent. We provide a proof-of-concept study on synthetic benchmarks to demonstrate the effectiveness of our approach.
title Decision Trees That Remember: Gradient-Based Learning of Recurrent Decision Trees with Memory
topic Machine Learning
url https://arxiv.org/abs/2502.04052