A Coding-Theoretic Analysis of Hyperspherical Prototypical Learning Geometry
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916694099755008 |
|---|---|
| author | Lindström, Martin Rodríguez-Gálvez, Borja Thobaben, Ragnar Skoglund, Mikael |
| author_facet | Lindström, Martin Rodríguez-Gálvez, Borja Thobaben, Ragnar Skoglund, Mikael |
| contents | Hyperspherical Prototypical Learning (HPL) is a supervised approach to representation learning that designs class prototypes on the unit hypersphere. The prototypes bias the representations to class separation in a scale invariant and known geometry. Previous approaches to HPL have either of the following shortcomings: (i) they follow an unprincipled optimisation procedure; or (ii) they are theoretically sound, but are constrained to only one possible latent dimension. In this paper, we address both shortcomings. To address (i), we present a principled optimisation procedure whose solution we show is optimal. To address (ii), we construct well-separated prototypes in a wide range of dimensions using linear block codes. Additionally, we give a full characterisation of the optimal prototype placement in terms of achievable and converse bounds, showing that our proposed methods are near-optimal. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_07664 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Coding-Theoretic Analysis of Hyperspherical Prototypical Learning Geometry Lindström, Martin Rodríguez-Gálvez, Borja Thobaben, Ragnar Skoglund, Mikael Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Signal Processing Hyperspherical Prototypical Learning (HPL) is a supervised approach to representation learning that designs class prototypes on the unit hypersphere. The prototypes bias the representations to class separation in a scale invariant and known geometry. Previous approaches to HPL have either of the following shortcomings: (i) they follow an unprincipled optimisation procedure; or (ii) they are theoretically sound, but are constrained to only one possible latent dimension. In this paper, we address both shortcomings. To address (i), we present a principled optimisation procedure whose solution we show is optimal. To address (ii), we construct well-separated prototypes in a wide range of dimensions using linear block codes. Additionally, we give a full characterisation of the optimal prototype placement in terms of achievable and converse bounds, showing that our proposed methods are near-optimal. |
| title | A Coding-Theoretic Analysis of Hyperspherical Prototypical Learning Geometry |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Signal Processing |
| url | https://arxiv.org/abs/2407.07664 |