HNCSE: Advancing Sentence Embeddings via Hybrid Contrastive Learning with Hard Negatives
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915024866377728 |
|---|---|
| author | Liu, Wenxiao Yang, Zihong Li, Chaozhuo Hong, Zijin Ma, Jianfeng Liu, Zhiquan Zhang, Litian Huang, Feiran |
| author_facet | Liu, Wenxiao Yang, Zihong Li, Chaozhuo Hong, Zijin Ma, Jianfeng Liu, Zhiquan Zhang, Litian Huang, Feiran |
| contents | Unsupervised sentence representation learning remains a critical challenge in modern natural language processing (NLP) research. Recently, contrastive learning techniques have achieved significant success in addressing this issue by effectively capturing textual semantics. Many such approaches prioritize the optimization using negative samples. In fields such as computer vision, hard negative samples (samples that are close to the decision boundary and thus more difficult to distinguish) have been shown to enhance representation learning. However, adapting hard negatives to contrastive sentence learning is complex due to the intricate syntactic and semantic details of text. To address this problem, we propose HNCSE, a novel contrastive learning framework that extends the leading SimCSE approach. The hallmark of HNCSE is its innovative use of hard negative samples to enhance the learning of both positive and negative samples, thereby achieving a deeper semantic understanding. Empirical tests on semantic textual similarity and transfer task datasets validate the superiority of HNCSE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_12156 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | HNCSE: Advancing Sentence Embeddings via Hybrid Contrastive Learning with Hard Negatives Liu, Wenxiao Yang, Zihong Li, Chaozhuo Hong, Zijin Ma, Jianfeng Liu, Zhiquan Zhang, Litian Huang, Feiran Computation and Language Artificial Intelligence Unsupervised sentence representation learning remains a critical challenge in modern natural language processing (NLP) research. Recently, contrastive learning techniques have achieved significant success in addressing this issue by effectively capturing textual semantics. Many such approaches prioritize the optimization using negative samples. In fields such as computer vision, hard negative samples (samples that are close to the decision boundary and thus more difficult to distinguish) have been shown to enhance representation learning. However, adapting hard negatives to contrastive sentence learning is complex due to the intricate syntactic and semantic details of text. To address this problem, we propose HNCSE, a novel contrastive learning framework that extends the leading SimCSE approach. The hallmark of HNCSE is its innovative use of hard negative samples to enhance the learning of both positive and negative samples, thereby achieving a deeper semantic understanding. Empirical tests on semantic textual similarity and transfer task datasets validate the superiority of HNCSE. |
| title | HNCSE: Advancing Sentence Embeddings via Hybrid Contrastive Learning with Hard Negatives |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2411.12156 |