Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2408.12936 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908415325896704 |
|---|---|
| author | Denoodt, Fabian de Boer, Bart Oramas, José |
| author_facet | Denoodt, Fabian de Boer, Bart Oramas, José |
| contents | We introduce Smooth InfoMax (SIM), a self-supervised representation learning method that incorporates interpretability constraints into the latent representations at different depths of the network. Based on $β$-VAEs, SIM's architecture consists of probabilistic modules optimized locally with the InfoNCE loss to produce Gaussian-distributed representations regularized toward the standard normal distribution. This creates smooth, well-defined, and better-disentangled latent spaces, enabling easier post-hoc analysis. Evaluated on speech data, SIM preserves the large-scale training benefits of Greedy InfoMax while improving the effectiveness of post-hoc interpretability methods across layers. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_12936 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Smooth InfoMax -- Towards Easier Post-Hoc Interpretability Denoodt, Fabian de Boer, Bart Oramas, José Machine Learning Artificial Intelligence We introduce Smooth InfoMax (SIM), a self-supervised representation learning method that incorporates interpretability constraints into the latent representations at different depths of the network. Based on $β$-VAEs, SIM's architecture consists of probabilistic modules optimized locally with the InfoNCE loss to produce Gaussian-distributed representations regularized toward the standard normal distribution. This creates smooth, well-defined, and better-disentangled latent spaces, enabling easier post-hoc analysis. Evaluated on speech data, SIM preserves the large-scale training benefits of Greedy InfoMax while improving the effectiveness of post-hoc interpretability methods across layers. |
| title | Smooth InfoMax -- Towards Easier Post-Hoc Interpretability |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2408.12936 |