Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2312.14254 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909219529162752 |
|---|---|
| author | Sristi, Ram Dyuthi Lindenbaum, Ofir Lifshitz, Shira Lavzin, Maria Schiller, Jackie Mishne, Gal Benisty, Hadas |
| author_facet | Sristi, Ram Dyuthi Lindenbaum, Ofir Lifshitz, Shira Lavzin, Maria Schiller, Jackie Mishne, Gal Benisty, Hadas |
| contents | Feature selection is a crucial tool in machine learning and is widely applied across various scientific disciplines. Traditional supervised methods generally identify a universal set of informative features for the entire population. However, feature relevance often varies with context, while the context itself may not directly affect the outcome variable. Here, we propose a novel architecture for contextual feature selection where the subset of selected features is conditioned on the value of context variables. Our new approach, Conditional Stochastic Gates (c-STG), models the importance of features using conditional Bernoulli variables whose parameters are predicted based on contextual variables. We introduce a hypernetwork that maps context variables to feature selection parameters to learn the context-dependent gates along with a prediction model. We further present a theoretical analysis of our model, indicating that it can improve performance and flexibility over population-level methods in complex feature selection settings. Finally, we conduct an extensive benchmark using simulated and real-world datasets across multiple domains demonstrating that c-STG can lead to improved feature selection capabilities while enhancing prediction accuracy and interpretability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_14254 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Contextual Feature Selection with Conditional Stochastic Gates Sristi, Ram Dyuthi Lindenbaum, Ofir Lifshitz, Shira Lavzin, Maria Schiller, Jackie Mishne, Gal Benisty, Hadas Machine Learning Neural and Evolutionary Computing Feature selection is a crucial tool in machine learning and is widely applied across various scientific disciplines. Traditional supervised methods generally identify a universal set of informative features for the entire population. However, feature relevance often varies with context, while the context itself may not directly affect the outcome variable. Here, we propose a novel architecture for contextual feature selection where the subset of selected features is conditioned on the value of context variables. Our new approach, Conditional Stochastic Gates (c-STG), models the importance of features using conditional Bernoulli variables whose parameters are predicted based on contextual variables. We introduce a hypernetwork that maps context variables to feature selection parameters to learn the context-dependent gates along with a prediction model. We further present a theoretical analysis of our model, indicating that it can improve performance and flexibility over population-level methods in complex feature selection settings. Finally, we conduct an extensive benchmark using simulated and real-world datasets across multiple domains demonstrating that c-STG can lead to improved feature selection capabilities while enhancing prediction accuracy and interpretability. |
| title | Contextual Feature Selection with Conditional Stochastic Gates |
| topic | Machine Learning Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2312.14254 |