Geometric Analysis of Unconstrained Feature Models with $d=K$
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913440151371776 |
|---|---|
| author | Shen, Yi Gu, Shao |
| author_facet | Shen, Yi Gu, Shao |
| contents | Recently, interesting empirical phenomena known as Neural Collapse have been observed during the final phase of training deep neural networks for classification tasks. We examine this issue when the feature dimension d is equal to the number of classes K. We demonstrate that two popular unconstrained feature models are strict saddle functions, with every critical point being either a global minimum or a strict saddle point that can be exited using negative curvatures. The primary findings conclusively confirm the conjecture on the unconstrained feature models in previous articles. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_10702 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Geometric Analysis of Unconstrained Feature Models with $d=K$ Shen, Yi Gu, Shao Machine Learning Recently, interesting empirical phenomena known as Neural Collapse have been observed during the final phase of training deep neural networks for classification tasks. We examine this issue when the feature dimension d is equal to the number of classes K. We demonstrate that two popular unconstrained feature models are strict saddle functions, with every critical point being either a global minimum or a strict saddle point that can be exited using negative curvatures. The primary findings conclusively confirm the conjecture on the unconstrained feature models in previous articles. |
| title | Geometric Analysis of Unconstrained Feature Models with $d=K$ |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2407.10702 |