TNCSE: Tensor's Norm Constraints for Unsupervised Contrastive Learning of Sentence Embeddings
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913739124506624 |
|---|---|
| author | Zong, Tianyu Shi, Bingkang Yi, Hongzhu Xu, Jungang |
| author_facet | Zong, Tianyu Shi, Bingkang Yi, Hongzhu Xu, Jungang |
| contents | Unsupervised sentence embedding representation has become a hot research topic in natural language processing. As a tensor, sentence embedding has two critical properties: direction and norm. Existing works have been limited to constraining only the orientation of the samples' representations while ignoring the features of their module lengths. To address this issue, we propose a new training objective that optimizes the training of unsupervised contrastive learning by constraining the module length features between positive samples. We combine the training objective of Tensor's Norm Constraints with ensemble learning to propose a new Sentence Embedding representation framework, TNCSE. We evaluate seven semantic text similarity tasks, and the results show that TNCSE and derived models are the current state-of-the-art approach; in addition, we conduct extensive zero-shot evaluations, and the results show that TNCSE outperforms other baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_12739 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TNCSE: Tensor's Norm Constraints for Unsupervised Contrastive Learning of Sentence Embeddings Zong, Tianyu Shi, Bingkang Yi, Hongzhu Xu, Jungang Computation and Language Artificial Intelligence Unsupervised sentence embedding representation has become a hot research topic in natural language processing. As a tensor, sentence embedding has two critical properties: direction and norm. Existing works have been limited to constraining only the orientation of the samples' representations while ignoring the features of their module lengths. To address this issue, we propose a new training objective that optimizes the training of unsupervised contrastive learning by constraining the module length features between positive samples. We combine the training objective of Tensor's Norm Constraints with ensemble learning to propose a new Sentence Embedding representation framework, TNCSE. We evaluate seven semantic text similarity tasks, and the results show that TNCSE and derived models are the current state-of-the-art approach; in addition, we conduct extensive zero-shot evaluations, and the results show that TNCSE outperforms other baselines. |
| title | TNCSE: Tensor's Norm Constraints for Unsupervised Contrastive Learning of Sentence Embeddings |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.12739 |