Translationese-index: Using Likelihood Ratios for Graded and Generalizable Measurement of Translationese
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916957861707776 |
|---|---|
| author | Liu, Yikang Zhang, Wanyang Wang, Yiming Tang, Jialong Zhang, Pei Yang, Baosong Huang, Fei Wang, Rui Hu, Hai |
| author_facet | Liu, Yikang Zhang, Wanyang Wang, Yiming Tang, Jialong Zhang, Pei Yang, Baosong Huang, Fei Wang, Rui Hu, Hai |
| contents | Translationese refers to linguistic properties that usually occur in translated texts. Previous works study translationese by framing it as a binary classification between original texts and translated texts. In this paper, we argue that translationese should be graded instead of binary and propose the first measure for translationese -- the translationese-index (T-index), computed from the likelihood ratios of two contrastively fine-tuned language models (LMs). We use synthesized translations and translations in the wild to evaluate T-index's generalizability in cross-domain settings and its validity against human judgments. Our results show that T-index can generalize to unseen genres, authors, and language pairs. Moreover, T-index computed using two 0.5B LMs fine-tuned on only 1-5k pairs of synthetic data can effectively capture translationese, as demonstrated by alignment with human pointwise ratings and pairwise judgments. Additionally, the correlation between T-index and existing machine translation (MT) quality estimation (QE) metrics such as BLEU and COMET is low, suggesting that T-index is not covered by these metrics and can serve as a complementary metric in MT QE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_12260 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Translationese-index: Using Likelihood Ratios for Graded and Generalizable Measurement of Translationese Liu, Yikang Zhang, Wanyang Wang, Yiming Tang, Jialong Zhang, Pei Yang, Baosong Huang, Fei Wang, Rui Hu, Hai Computation and Language Translationese refers to linguistic properties that usually occur in translated texts. Previous works study translationese by framing it as a binary classification between original texts and translated texts. In this paper, we argue that translationese should be graded instead of binary and propose the first measure for translationese -- the translationese-index (T-index), computed from the likelihood ratios of two contrastively fine-tuned language models (LMs). We use synthesized translations and translations in the wild to evaluate T-index's generalizability in cross-domain settings and its validity against human judgments. Our results show that T-index can generalize to unseen genres, authors, and language pairs. Moreover, T-index computed using two 0.5B LMs fine-tuned on only 1-5k pairs of synthetic data can effectively capture translationese, as demonstrated by alignment with human pointwise ratings and pairwise judgments. Additionally, the correlation between T-index and existing machine translation (MT) quality estimation (QE) metrics such as BLEU and COMET is low, suggesting that T-index is not covered by these metrics and can serve as a complementary metric in MT QE. |
| title | Translationese-index: Using Likelihood Ratios for Graded and Generalizable Measurement of Translationese |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2507.12260 |