Indirectly Parameterized Concrete Autoencoders
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914913774993408 |
|---|---|
| author | Nilsson, Alfred Wijk, Klas Gutha, Sai bharath chandra Englesson, Erik Hotti, Alexandra Saccardi, Carlo Kviman, Oskar Lagergren, Jens Vinuesa, Ricardo Azizpour, Hossein |
| author_facet | Nilsson, Alfred Wijk, Klas Gutha, Sai bharath chandra Englesson, Erik Hotti, Alexandra Saccardi, Carlo Kviman, Oskar Lagergren, Jens Vinuesa, Ricardo Azizpour, Hossein |
| contents | Feature selection is a crucial task in settings where data is high-dimensional or acquiring the full set of features is costly. Recent developments in neural network-based embedded feature selection show promising results across a wide range of applications. Concrete Autoencoders (CAEs), considered state-of-the-art in embedded feature selection, may struggle to achieve stable joint optimization, hurting their training time and generalization. In this work, we identify that this instability is correlated with the CAE learning duplicate selections. To remedy this, we propose a simple and effective improvement: Indirectly Parameterized CAEs (IP-CAEs). IP-CAEs learn an embedding and a mapping from it to the Gumbel-Softmax distributions' parameters. Despite being simple to implement, IP-CAE exhibits significant and consistent improvements over CAE in both generalization and training time across several datasets for reconstruction and classification. Unlike CAE, IP-CAE effectively leverages non-linear relationships and does not require retraining the jointly optimized decoder. Furthermore, our approach is, in principle, generalizable to Gumbel-Softmax distributions beyond feature selection. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_00563 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Indirectly Parameterized Concrete Autoencoders Nilsson, Alfred Wijk, Klas Gutha, Sai bharath chandra Englesson, Erik Hotti, Alexandra Saccardi, Carlo Kviman, Oskar Lagergren, Jens Vinuesa, Ricardo Azizpour, Hossein Machine Learning Feature selection is a crucial task in settings where data is high-dimensional or acquiring the full set of features is costly. Recent developments in neural network-based embedded feature selection show promising results across a wide range of applications. Concrete Autoencoders (CAEs), considered state-of-the-art in embedded feature selection, may struggle to achieve stable joint optimization, hurting their training time and generalization. In this work, we identify that this instability is correlated with the CAE learning duplicate selections. To remedy this, we propose a simple and effective improvement: Indirectly Parameterized CAEs (IP-CAEs). IP-CAEs learn an embedding and a mapping from it to the Gumbel-Softmax distributions' parameters. Despite being simple to implement, IP-CAE exhibits significant and consistent improvements over CAE in both generalization and training time across several datasets for reconstruction and classification. Unlike CAE, IP-CAE effectively leverages non-linear relationships and does not require retraining the jointly optimized decoder. Furthermore, our approach is, in principle, generalizable to Gumbel-Softmax distributions beyond feature selection. |
| title | Indirectly Parameterized Concrete Autoencoders |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2403.00563 |