Refining Sentence Embedding Model through Ranking Sentences Generation with Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912412472442880 |
|---|---|
| author | He, Liyang Liu, Chenglong Li, Rui Huang, Zhenya Ruan, Shulan Zhou, Jun Chen, Enhong |
| author_facet | He, Liyang Liu, Chenglong Li, Rui Huang, Zhenya Ruan, Shulan Zhou, Jun Chen, Enhong |
| contents | Sentence embedding is essential for many NLP tasks, with contrastive learning methods achieving strong performance using annotated datasets like NLI. Yet, the reliance on manual labels limits scalability. Recent studies leverage large language models (LLMs) to generate sentence pairs, reducing annotation dependency. However, they overlook ranking information crucial for fine-grained semantic distinctions. To tackle this challenge, we propose a method for controlling the generation direction of LLMs in the latent space. Unlike unconstrained generation, the controlled approach ensures meaningful semantic divergence. Then, we refine exist sentence embedding model by integrating ranking information and semantic information. Experiments on multiple benchmarks demonstrate that our method achieves new SOTA performance with a modest cost in ranking sentence synthesis. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_13656 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Refining Sentence Embedding Model through Ranking Sentences Generation with Large Language Models He, Liyang Liu, Chenglong Li, Rui Huang, Zhenya Ruan, Shulan Zhou, Jun Chen, Enhong Computation and Language Sentence embedding is essential for many NLP tasks, with contrastive learning methods achieving strong performance using annotated datasets like NLI. Yet, the reliance on manual labels limits scalability. Recent studies leverage large language models (LLMs) to generate sentence pairs, reducing annotation dependency. However, they overlook ranking information crucial for fine-grained semantic distinctions. To tackle this challenge, we propose a method for controlling the generation direction of LLMs in the latent space. Unlike unconstrained generation, the controlled approach ensures meaningful semantic divergence. Then, we refine exist sentence embedding model by integrating ranking information and semantic information. Experiments on multiple benchmarks demonstrate that our method achieves new SOTA performance with a modest cost in ranking sentence synthesis. |
| title | Refining Sentence Embedding Model through Ranking Sentences Generation with Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2502.13656 |