Isotropy, Clusters, and Classifiers
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929358856257536 |
|---|---|
| author | Mickus, Timothee Grönroos, Stig-Arne Attieh, Joseph |
| author_facet | Mickus, Timothee Grönroos, Stig-Arne Attieh, Joseph |
| contents | Whether embedding spaces use all their dimensions equally, i.e., whether they are isotropic, has been a recent subject of discussion. Evidence has been accrued both for and against enforcing isotropy in embedding spaces. In the present paper, we stress that isotropy imposes requirements on the embedding space that are not compatible with the presence of clusters -- which also negatively impacts linear classification objectives. We demonstrate this fact both mathematically and empirically and use it to shed light on previous results from the literature. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_03191 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Isotropy, Clusters, and Classifiers Mickus, Timothee Grönroos, Stig-Arne Attieh, Joseph Machine Learning Computation and Language Whether embedding spaces use all their dimensions equally, i.e., whether they are isotropic, has been a recent subject of discussion. Evidence has been accrued both for and against enforcing isotropy in embedding spaces. In the present paper, we stress that isotropy imposes requirements on the embedding space that are not compatible with the presence of clusters -- which also negatively impacts linear classification objectives. We demonstrate this fact both mathematically and empirically and use it to shed light on previous results from the literature. |
| title | Isotropy, Clusters, and Classifiers |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2402.03191 |