Saved in:
Bibliographic Details
Main Authors: Mickus, Timothee, Grönroos, Stig-Arne, Attieh, Joseph
Format: Preprint
Published: 2024
Subjects:
Online Access:https://arxiv.org/abs/2402.03191
Tags: Add Tag
No Tags, Be the first to tag this record!
Table of Contents:
  • Whether embedding spaces use all their dimensions equally, i.e., whether they are isotropic, has been a recent subject of discussion. Evidence has been accrued both for and against enforcing isotropy in embedding spaces. In the present paper, we stress that isotropy imposes requirements on the embedding space that are not compatible with the presence of clusters -- which also negatively impacts linear classification objectives. We demonstrate this fact both mathematically and empirically and use it to shed light on previous results from the literature.