Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2309.12697 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929211908816896 |
|---|---|
| author | Herbold, Steffen |
| author_facet | Herbold, Steffen |
| contents | Semantic similarity between natural language texts is typically measured either by looking at the overlap between subsequences (e.g., BLEU) or by using embeddings (e.g., BERTScore, S-BERT). Within this paper, we argue that when we are only interested in measuring the semantic similarity, it is better to directly predict the similarity using a fine-tuned model for such a task. Using a fine-tuned model for the Semantic Textual Similarity Benchmark tasks (STS-B) from the GLUE benchmark, we define the STSScore approach and show that the resulting similarity is better aligned with our expectations on a robust semantic similarity measure than other approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_12697 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Semantic similarity prediction is better than other semantic similarity measures Herbold, Steffen Computation and Language Machine Learning Semantic similarity between natural language texts is typically measured either by looking at the overlap between subsequences (e.g., BLEU) or by using embeddings (e.g., BERTScore, S-BERT). Within this paper, we argue that when we are only interested in measuring the semantic similarity, it is better to directly predict the similarity using a fine-tuned model for such a task. Using a fine-tuned model for the Semantic Textual Similarity Benchmark tasks (STS-B) from the GLUE benchmark, we define the STSScore approach and show that the resulting similarity is better aligned with our expectations on a robust semantic similarity measure than other approaches. |
| title | Semantic similarity prediction is better than other semantic similarity measures |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2309.12697 |