Neural collapse in the orthoplex regime
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908903434878976 |
|---|---|
| author | Alcala, James Andreeva, Rayna Kobzar, Vladimir A. Mixon, Dustin G. Na, Sanghoon Sule, Shashank Xie, Yangxinyu |
| author_facet | Alcala, James Andreeva, Rayna Kobzar, Vladimir A. Mixon, Dustin G. Na, Sanghoon Sule, Shashank Xie, Yangxinyu |
| contents | When training a neural network for classification, the feature vectors of the training set are known to collapse to the vertices of a regular simplex, provided the dimension $d$ of the feature space and the number $n$ of classes satisfies $n\leq d+1$. This phenomenon is known as neural collapse. For other applications like language models, one instead takes $n\gg d$. Here, the neural collapse phenomenon still occurs, but with different emergent geometric figures. We characterize these geometric figures in the orthoplex regime where $d+2\leq n\leq 2d$. The techniques in our analysis primarily involve Radon's theorem and convexity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_20587 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Neural collapse in the orthoplex regime Alcala, James Andreeva, Rayna Kobzar, Vladimir A. Mixon, Dustin G. Na, Sanghoon Sule, Shashank Xie, Yangxinyu Machine Learning Information Theory Metric Geometry When training a neural network for classification, the feature vectors of the training set are known to collapse to the vertices of a regular simplex, provided the dimension $d$ of the feature space and the number $n$ of classes satisfies $n\leq d+1$. This phenomenon is known as neural collapse. For other applications like language models, one instead takes $n\gg d$. Here, the neural collapse phenomenon still occurs, but with different emergent geometric figures. We characterize these geometric figures in the orthoplex regime where $d+2\leq n\leq 2d$. The techniques in our analysis primarily involve Radon's theorem and convexity. |
| title | Neural collapse in the orthoplex regime |
| topic | Machine Learning Information Theory Metric Geometry |
| url | https://arxiv.org/abs/2603.20587 |