Probabilistic Contrastive Learning with Explicit Concentration on the Hypersphere
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916261131190272 |
|---|---|
| author | Li, Hongwei Bran Ouyang, Cheng Amiranashvili, Tamaz Rosen, Matthew S. Menze, Bjoern Iglesias, Juan Eugenio |
| author_facet | Li, Hongwei Bran Ouyang, Cheng Amiranashvili, Tamaz Rosen, Matthew S. Menze, Bjoern Iglesias, Juan Eugenio |
| contents | Self-supervised contrastive learning has predominantly adopted deterministic methods, which are not suited for environments characterized by uncertainty and noise. This paper introduces a new perspective on incorporating uncertainty into contrastive learning by embedding representations within a spherical space, inspired by the von Mises-Fisher distribution (vMF). We introduce an unnormalized form of vMF and leverage the concentration parameter, kappa, as a direct, interpretable measure to quantify uncertainty explicitly. This approach not only provides a probabilistic interpretation of the embedding space but also offers a method to calibrate model confidence against varying levels of data corruption and characteristics. Our empirical results demonstrate that the estimated concentration parameter correlates strongly with the degree of unforeseen data corruption encountered at test time, enables failure analysis, and enhances existing out-of-distribution detection methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_16460 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Probabilistic Contrastive Learning with Explicit Concentration on the Hypersphere Li, Hongwei Bran Ouyang, Cheng Amiranashvili, Tamaz Rosen, Matthew S. Menze, Bjoern Iglesias, Juan Eugenio Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Self-supervised contrastive learning has predominantly adopted deterministic methods, which are not suited for environments characterized by uncertainty and noise. This paper introduces a new perspective on incorporating uncertainty into contrastive learning by embedding representations within a spherical space, inspired by the von Mises-Fisher distribution (vMF). We introduce an unnormalized form of vMF and leverage the concentration parameter, kappa, as a direct, interpretable measure to quantify uncertainty explicitly. This approach not only provides a probabilistic interpretation of the embedding space but also offers a method to calibrate model confidence against varying levels of data corruption and characteristics. Our empirical results demonstrate that the estimated concentration parameter correlates strongly with the degree of unforeseen data corruption encountered at test time, enables failure analysis, and enhances existing out-of-distribution detection methods. |
| title | Probabilistic Contrastive Learning with Explicit Concentration on the Hypersphere |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.16460 |