Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2305.11475 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912694234251264 |
|---|---|
| author | Siems, Julien Ditschuneit, Konstantin Ripken, Winfried Lindborg, Alma Schambach, Maximilian Otterbach, Johannes S. Genzel, Martin |
| author_facet | Siems, Julien Ditschuneit, Konstantin Ripken, Winfried Lindborg, Alma Schambach, Maximilian Otterbach, Johannes S. Genzel, Martin |
| contents | Generalized Additive Models (GAMs) have recently experienced a resurgence in popularity due to their interpretability, which arises from expressing the target value as a sum of non-linear transformations of the features. Despite the current enthusiasm for GAMs, their susceptibility to concurvity - i.e., (possibly non-linear) dependencies between the features - has hitherto been largely overlooked. Here, we demonstrate how concurvity can severly impair the interpretability of GAMs and propose a remedy: a conceptually simple, yet effective regularizer which penalizes pairwise correlations of the non-linearly transformed feature variables. This procedure is applicable to any differentiable additive model, such as Neural Additive Models or NeuralProphet, and enhances interpretability by eliminating ambiguities due to self-canceling feature contributions. We validate the effectiveness of our regularizer in experiments on synthetic as well as real-world datasets for time-series and tabular data. Our experiments show that concurvity in GAMs can be reduced without significantly compromising prediction quality, improving interpretability and reducing variance in the feature importances. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_11475 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Curve Your Enthusiasm: Concurvity Regularization in Differentiable Generalized Additive Models Siems, Julien Ditschuneit, Konstantin Ripken, Winfried Lindborg, Alma Schambach, Maximilian Otterbach, Johannes S. Genzel, Martin Machine Learning Generalized Additive Models (GAMs) have recently experienced a resurgence in popularity due to their interpretability, which arises from expressing the target value as a sum of non-linear transformations of the features. Despite the current enthusiasm for GAMs, their susceptibility to concurvity - i.e., (possibly non-linear) dependencies between the features - has hitherto been largely overlooked. Here, we demonstrate how concurvity can severly impair the interpretability of GAMs and propose a remedy: a conceptually simple, yet effective regularizer which penalizes pairwise correlations of the non-linearly transformed feature variables. This procedure is applicable to any differentiable additive model, such as Neural Additive Models or NeuralProphet, and enhances interpretability by eliminating ambiguities due to self-canceling feature contributions. We validate the effectiveness of our regularizer in experiments on synthetic as well as real-world datasets for time-series and tabular data. Our experiments show that concurvity in GAMs can be reduced without significantly compromising prediction quality, improving interpretability and reducing variance in the feature importances. |
| title | Curve Your Enthusiasm: Concurvity Regularization in Differentiable Generalized Additive Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2305.11475 |