reCSE: Portable Reshaping Features for Sentence Embedding in Self-supervised Contrastive Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909295648440320 |
|---|---|
| author | Zhao, Fufangchen Gao, Jian Yan, Danfeng |
| author_facet | Zhao, Fufangchen Gao, Jian Yan, Danfeng |
| contents | We propose reCSE, a self supervised contrastive learning sentence representation framework based on feature reshaping. This framework is different from the current advanced models that use discrete data augmentation methods, but instead reshapes the input features of the original sentence, aggregates the global information of each token in the sentence, and alleviates the common problems of representation polarity and GPU memory consumption linear increase in current advanced models. In addition, our reCSE has achieved competitive performance in semantic similarity tasks. And the experiment proves that our proposed feature reshaping method has strong universality, which can be transplanted to other self supervised contrastive learning frameworks and enhance their representation ability, even achieving state-of-the-art performance. Our code is available at https://github.com/heavenhellchen/reCSE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_04975 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | reCSE: Portable Reshaping Features for Sentence Embedding in Self-supervised Contrastive Learning Zhao, Fufangchen Gao, Jian Yan, Danfeng Computation and Language We propose reCSE, a self supervised contrastive learning sentence representation framework based on feature reshaping. This framework is different from the current advanced models that use discrete data augmentation methods, but instead reshapes the input features of the original sentence, aggregates the global information of each token in the sentence, and alleviates the common problems of representation polarity and GPU memory consumption linear increase in current advanced models. In addition, our reCSE has achieved competitive performance in semantic similarity tasks. And the experiment proves that our proposed feature reshaping method has strong universality, which can be transplanted to other self supervised contrastive learning frameworks and enhance their representation ability, even achieving state-of-the-art performance. Our code is available at https://github.com/heavenhellchen/reCSE. |
| title | reCSE: Portable Reshaping Features for Sentence Embedding in Self-supervised Contrastive Learning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2408.04975 |