Partial Information Decomposition for Data Interpretability and Feature Selection
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914157903740928 |
|---|---|
| author | Westphal, Charles Hailes, Stephen Musolesi, Mirco |
| author_facet | Westphal, Charles Hailes, Stephen Musolesi, Mirco |
| contents | In this paper, we introduce Partial Information Decomposition of Features (PIDF), a new paradigm for simultaneous data interpretability and feature selection. Contrary to traditional methods that assign a single importance value, our approach is based on three metrics per feature: the mutual information shared with the target variable, the feature's contribution to synergistic information, and the amount of this information that is redundant. In particular, we develop a novel procedure based on these three metrics, which reveals not only how features are correlated with the target but also the additional and overlapping information provided by considering them in combination with other features. We extensively evaluate PIDF using both synthetic and real-world data, demonstrating its potential applications and effectiveness, by considering case studies from genetics and neuroscience. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_19212 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Partial Information Decomposition for Data Interpretability and Feature Selection Westphal, Charles Hailes, Stephen Musolesi, Mirco Machine Learning Artificial Intelligence Information Theory In this paper, we introduce Partial Information Decomposition of Features (PIDF), a new paradigm for simultaneous data interpretability and feature selection. Contrary to traditional methods that assign a single importance value, our approach is based on three metrics per feature: the mutual information shared with the target variable, the feature's contribution to synergistic information, and the amount of this information that is redundant. In particular, we develop a novel procedure based on these three metrics, which reveals not only how features are correlated with the target but also the additional and overlapping information provided by considering them in combination with other features. We extensively evaluate PIDF using both synthetic and real-world data, demonstrating its potential applications and effectiveness, by considering case studies from genetics and neuroscience. |
| title | Partial Information Decomposition for Data Interpretability and Feature Selection |
| topic | Machine Learning Artificial Intelligence Information Theory |
| url | https://arxiv.org/abs/2405.19212 |