MINERVA: Mutual Information Neural Estimation for Supervised Feature Selection
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911191739138048 |
|---|---|
| author | Muvunza, Taurai Kraev, Egor Planell-Morell, Pere Shestopaloff, Alexander Y. |
| author_facet | Muvunza, Taurai Kraev, Egor Planell-Morell, Pere Shestopaloff, Alexander Y. |
| contents | Existing feature filters rely on statistical pair-wise dependence metrics to model feature-target relationships, but this approach may fail when the target depends on higher-order feature interactions rather than individual contributions. We introduce Mutual Information Neural Estimation Regularized Vetting Algorithm (MINERVA), a novel approach to supervised feature selection based on neural estimation of mutual information between features and targets. We paramaterize the approximation of mutual information with neural networks and perform feature selection using a carefully designed loss function augmented with sparsity-inducing regularizers. Our method is implemented in a two-stage process to decouple representation learning from feature selection, ensuring better generalization and a more accurate expression of feature importance. We present examples of ubiquitous dependency structures that are rarely captured in literature and show that our proposed method effectively captures these complex feature-target relationships by evaluating feature subsets as an ensemble. Experimental results on synthetic and real-life fraud datasets demonstrate the efficacy of our method and its ability to perform exact solutions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_02610 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MINERVA: Mutual Information Neural Estimation for Supervised Feature Selection Muvunza, Taurai Kraev, Egor Planell-Morell, Pere Shestopaloff, Alexander Y. Machine Learning Artificial Intelligence I.2.6; I.5.1; G.3 Existing feature filters rely on statistical pair-wise dependence metrics to model feature-target relationships, but this approach may fail when the target depends on higher-order feature interactions rather than individual contributions. We introduce Mutual Information Neural Estimation Regularized Vetting Algorithm (MINERVA), a novel approach to supervised feature selection based on neural estimation of mutual information between features and targets. We paramaterize the approximation of mutual information with neural networks and perform feature selection using a carefully designed loss function augmented with sparsity-inducing regularizers. Our method is implemented in a two-stage process to decouple representation learning from feature selection, ensuring better generalization and a more accurate expression of feature importance. We present examples of ubiquitous dependency structures that are rarely captured in literature and show that our proposed method effectively captures these complex feature-target relationships by evaluating feature subsets as an ensemble. Experimental results on synthetic and real-life fraud datasets demonstrate the efficacy of our method and its ability to perform exact solutions. |
| title | MINERVA: Mutual Information Neural Estimation for Supervised Feature Selection |
| topic | Machine Learning Artificial Intelligence I.2.6; I.5.1; G.3 |
| url | https://arxiv.org/abs/2510.02610 |