Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.17844 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913855651708928 |
|---|---|
| author | Deuser, Fabian Hausenblas, Philipp Schieber, Hannah Roth, Daniel Werner, Martin Oswald, Norbert |
| author_facet | Deuser, Fabian Hausenblas, Philipp Schieber, Hannah Roth, Daniel Werner, Martin Oswald, Norbert |
| contents | Contrastive learning is a representational learning paradigm in which a neural network maps data elements to feature vectors. It improves the feature space by forming lots with an anchor and examples that are either positive or negative based on class similarity. Hard negative examples, which are close to the anchor in the feature space but from a different class, improve learning performance. Finding such examples of high quality efficiently in large, high-dimensional datasets is computationally challenging. In this paper, we propose a GPU-friendly Locality-Sensitive Hashing (LSH) scheme that quantizes real-valued feature vectors into binary representations for approximate nearest neighbor search. We investigate its theoretical properties and evaluate it on several datasets from textual and visual domain. Our approach achieves comparable or better performance while requiring significantly less computation than existing hard negative mining strategies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_17844 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Locality-Sensitive Hashing for Efficient Hard Negative Sampling in Contrastive Learning Deuser, Fabian Hausenblas, Philipp Schieber, Hannah Roth, Daniel Werner, Martin Oswald, Norbert Computer Vision and Pattern Recognition Contrastive learning is a representational learning paradigm in which a neural network maps data elements to feature vectors. It improves the feature space by forming lots with an anchor and examples that are either positive or negative based on class similarity. Hard negative examples, which are close to the anchor in the feature space but from a different class, improve learning performance. Finding such examples of high quality efficiently in large, high-dimensional datasets is computationally challenging. In this paper, we propose a GPU-friendly Locality-Sensitive Hashing (LSH) scheme that quantizes real-valued feature vectors into binary representations for approximate nearest neighbor search. We investigate its theoretical properties and evaluate it on several datasets from textual and visual domain. Our approach achieves comparable or better performance while requiring significantly less computation than existing hard negative mining strategies. |
| title | Locality-Sensitive Hashing for Efficient Hard Negative Sampling in Contrastive Learning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.17844 |