Radial-VCReg: More Informative Representation Learning Through Radial Gaussianization
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918339955130368 |
|---|---|
| author | Kuang, Yilun Dagade, Yash Chakraborty, Deep Learned-Miller, Erik Balestriero, Randall Rudner, Tim G. J. LeCun, Yann |
| author_facet | Kuang, Yilun Dagade, Yash Chakraborty, Deep Learned-Miller, Erik Balestriero, Randall Rudner, Tim G. J. LeCun, Yann |
| contents | Self-supervised learning aims to learn maximally informative representations, but explicit information maximization is hindered by the curse of dimensionality. Existing methods like VCReg address this by regularizing first and second-order feature statistics, which cannot fully achieve maximum entropy. We propose Radial-VCReg, which augments VCReg with a radial Gaussianization loss that aligns feature norms with the Chi distribution-a defining property of high-dimensional Gaussians. We prove that Radial-VCReg transforms a broader class of distributions towards normality compared to VCReg and show on synthetic and real-world datasets that it consistently improves performance by reducing higher-order dependencies and promoting more diverse and informative representations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_14272 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Radial-VCReg: More Informative Representation Learning Through Radial Gaussianization Kuang, Yilun Dagade, Yash Chakraborty, Deep Learned-Miller, Erik Balestriero, Randall Rudner, Tim G. J. LeCun, Yann Machine Learning Self-supervised learning aims to learn maximally informative representations, but explicit information maximization is hindered by the curse of dimensionality. Existing methods like VCReg address this by regularizing first and second-order feature statistics, which cannot fully achieve maximum entropy. We propose Radial-VCReg, which augments VCReg with a radial Gaussianization loss that aligns feature norms with the Chi distribution-a defining property of high-dimensional Gaussians. We prove that Radial-VCReg transforms a broader class of distributions towards normality compared to VCReg and show on synthetic and real-world datasets that it consistently improves performance by reducing higher-order dependencies and promoting more diverse and informative representations. |
| title | Radial-VCReg: More Informative Representation Learning Through Radial Gaussianization |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2602.14272 |