KerJEPA: Kernel Discrepancies for Euclidean Self-Supervised Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912783119941632 |
|---|---|
| author | Zimmermann, Eric Wiltzer, Harley Szeto, Justin Alvarez-Melis, David Mackey, Lester |
| author_facet | Zimmermann, Eric Wiltzer, Harley Szeto, Justin Alvarez-Melis, David Mackey, Lester |
| contents | Recent breakthroughs in self-supervised Joint-Embedding Predictive Architectures (JEPAs) have established that regularizing Euclidean representations toward isotropic Gaussian priors yields provable gains in training stability and downstream generalization. We introduce a new, flexible family of KerJEPAs, self-supervised learning algorithms with kernel-based regularizers. One instance of this family corresponds to the recently-introduced LeJEPA Epps-Pulley regularizer which approximates a sliced maximum mean discrepancy (MMD) with a Gaussian prior and Gaussian kernel. By expanding the class of viable kernels and priors and computing the closed-form high-dimensional limit of sliced MMDs, we develop alternative KerJEPAs with a number of favorable properties including improved training stability and design flexibility. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_19605 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | KerJEPA: Kernel Discrepancies for Euclidean Self-Supervised Learning Zimmermann, Eric Wiltzer, Harley Szeto, Justin Alvarez-Melis, David Mackey, Lester Machine Learning Computer Vision and Pattern Recognition Recent breakthroughs in self-supervised Joint-Embedding Predictive Architectures (JEPAs) have established that regularizing Euclidean representations toward isotropic Gaussian priors yields provable gains in training stability and downstream generalization. We introduce a new, flexible family of KerJEPAs, self-supervised learning algorithms with kernel-based regularizers. One instance of this family corresponds to the recently-introduced LeJEPA Epps-Pulley regularizer which approximates a sliced maximum mean discrepancy (MMD) with a Gaussian prior and Gaussian kernel. By expanding the class of viable kernels and priors and computing the closed-form high-dimensional limit of sliced MMDs, we develop alternative KerJEPAs with a number of favorable properties including improved training stability and design flexibility. |
| title | KerJEPA: Kernel Discrepancies for Euclidean Self-Supervised Learning |
| topic | Machine Learning Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.19605 |