Improving the quality of Web-mined Parallel Corpora of Low-Resource Languages using Debiasing Heuristics
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908548661772288 |
|---|---|
| author | Fernando, Aloka de Silva, Nisansa Velyuthan, Menan Rathnayake, Charitha Ranathunga, Surangika |
| author_facet | Fernando, Aloka de Silva, Nisansa Velyuthan, Menan Rathnayake, Charitha Ranathunga, Surangika |
| contents | Parallel Data Curation (PDC) techniques aim to filter out noisy parallel sentences from web-mined corpora. Ranking sentence pairs using similarity scores on sentence embeddings derived from Pre-trained Multilingual Language Models (multiPLMs) is the most common PDC technique. However, previous research has shown that the choice of the multiPLM significantly impacts the quality of the filtered parallel corpus, and the Neural Machine Translation (NMT) models trained using such data show a disparity across multiPLMs. This paper shows that this disparity is due to different multiPLMs being biased towards certain types of sentence pairs, which are treated as noise from an NMT point of view. We show that such noisy parallel sentences can be removed to a certain extent by employing a series of heuristics. The NMT models, trained using the curated corpus, lead to producing better results while minimizing the disparity across multiPLMs. We publicly release the source code and the curated datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_19074 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Improving the quality of Web-mined Parallel Corpora of Low-Resource Languages using Debiasing Heuristics Fernando, Aloka de Silva, Nisansa Velyuthan, Menan Rathnayake, Charitha Ranathunga, Surangika Computation and Language Parallel Data Curation (PDC) techniques aim to filter out noisy parallel sentences from web-mined corpora. Ranking sentence pairs using similarity scores on sentence embeddings derived from Pre-trained Multilingual Language Models (multiPLMs) is the most common PDC technique. However, previous research has shown that the choice of the multiPLM significantly impacts the quality of the filtered parallel corpus, and the Neural Machine Translation (NMT) models trained using such data show a disparity across multiPLMs. This paper shows that this disparity is due to different multiPLMs being biased towards certain types of sentence pairs, which are treated as noise from an NMT point of view. We show that such noisy parallel sentences can be removed to a certain extent by employing a series of heuristics. The NMT models, trained using the curated corpus, lead to producing better results while minimizing the disparity across multiPLMs. We publicly release the source code and the curated datasets. |
| title | Improving the quality of Web-mined Parallel Corpora of Low-Resource Languages using Debiasing Heuristics |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2502.19074 |