Improving Neural Additive Models with Bayesian Principles
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909365908275200 |
|---|---|
| author | Bouchiat, Kouroche Immer, Alexander Yèche, Hugo Rätsch, Gunnar Fortuin, Vincent |
| author_facet | Bouchiat, Kouroche Immer, Alexander Yèche, Hugo Rätsch, Gunnar Fortuin, Vincent |
| contents | Neural additive models (NAMs) enhance the transparency of deep neural networks by handling input features in separate additive sub-networks. However, they lack inherent mechanisms that provide calibrated uncertainties and enable selection of relevant features and interactions. Approaching NAMs from a Bayesian perspective, we augment them in three primary ways, namely by a) providing credible intervals for the individual additive sub-networks; b) estimating the marginal likelihood to perform an implicit selection of features via an empirical Bayes procedure; and c) facilitating the ranking of feature pairs as candidates for second-order interaction in fine-tuned models. In particular, we develop Laplace-approximated NAMs (LA-NAMs), which show improved empirical performance on tabular datasets and challenging real-world medical tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_16905 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Improving Neural Additive Models with Bayesian Principles Bouchiat, Kouroche Immer, Alexander Yèche, Hugo Rätsch, Gunnar Fortuin, Vincent Machine Learning Neural additive models (NAMs) enhance the transparency of deep neural networks by handling input features in separate additive sub-networks. However, they lack inherent mechanisms that provide calibrated uncertainties and enable selection of relevant features and interactions. Approaching NAMs from a Bayesian perspective, we augment them in three primary ways, namely by a) providing credible intervals for the individual additive sub-networks; b) estimating the marginal likelihood to perform an implicit selection of features via an empirical Bayes procedure; and c) facilitating the ranking of feature pairs as candidates for second-order interaction in fine-tuned models. In particular, we develop Laplace-approximated NAMs (LA-NAMs), which show improved empirical performance on tabular datasets and challenging real-world medical tasks. |
| title | Improving Neural Additive Models with Bayesian Principles |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2305.16905 |