Estimating Global Input Relevance and Enforcing Sparse Representations with a Scalable Spectral Neural Network Approach
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915630155825152 |
|---|---|
| author | Chicchi, Lorenzo Buffoni, Lorenzo Febbe, Diego Giambagli, Lorenzo Marino, Raffaele Fanelli, Duccio |
| author_facet | Chicchi, Lorenzo Buffoni, Lorenzo Febbe, Diego Giambagli, Lorenzo Marino, Raffaele Fanelli, Duccio |
| contents | In machine learning practice it is often useful to identify relevant input features. Isolating key input elements, ranked according their respective degree of relevance, can help to elaborate on the process of decision making. Here, we propose a novel method to estimate the relative importance of the input components for a Deep Neural Network. This is achieved by leveraging on a spectral re-parametrization of the optimization process. Eigenvalues associated to input nodes provide in fact a robust proxy to gauge the relevance of the supplied entry features. Notably, the spectral features ranking is performed automatically, as a byproduct of the network training, with no additional processing to be carried out. Moreover, by leveraging on the regularization of the eigenvalues, it is possible to enforce solutions making use of a minimum subset of the input components, increasing the explainability of the model and providing sparse input representations. The technique is compared to the most common methods in the literature and is successfully challenged against both synthetic and real data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_01183 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Estimating Global Input Relevance and Enforcing Sparse Representations with a Scalable Spectral Neural Network Approach Chicchi, Lorenzo Buffoni, Lorenzo Febbe, Diego Giambagli, Lorenzo Marino, Raffaele Fanelli, Duccio Machine Learning Disordered Systems and Neural Networks Statistical Mechanics Artificial Intelligence In machine learning practice it is often useful to identify relevant input features. Isolating key input elements, ranked according their respective degree of relevance, can help to elaborate on the process of decision making. Here, we propose a novel method to estimate the relative importance of the input components for a Deep Neural Network. This is achieved by leveraging on a spectral re-parametrization of the optimization process. Eigenvalues associated to input nodes provide in fact a robust proxy to gauge the relevance of the supplied entry features. Notably, the spectral features ranking is performed automatically, as a byproduct of the network training, with no additional processing to be carried out. Moreover, by leveraging on the regularization of the eigenvalues, it is possible to enforce solutions making use of a minimum subset of the input components, increasing the explainability of the model and providing sparse input representations. The technique is compared to the most common methods in the literature and is successfully challenged against both synthetic and real data. |
| title | Estimating Global Input Relevance and Enforcing Sparse Representations with a Scalable Spectral Neural Network Approach |
| topic | Machine Learning Disordered Systems and Neural Networks Statistical Mechanics Artificial Intelligence |
| url | https://arxiv.org/abs/2406.01183 |