XNLIeu: a dataset for cross-lingual NLI in Basque
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917636203347968 |
|---|---|
| author | Heredia, Maite Etxaniz, Julen Zulaika, Muitze Saralegi, Xabier Barnes, Jeremy Soroa, Aitor |
| author_facet | Heredia, Maite Etxaniz, Julen Zulaika, Muitze Saralegi, Xabier Barnes, Jeremy Soroa, Aitor |
| contents | XNLI is a popular Natural Language Inference (NLI) benchmark widely used to evaluate cross-lingual Natural Language Understanding (NLU) capabilities across languages. In this paper, we expand XNLI to include Basque, a low-resource language that can greatly benefit from transfer-learning approaches. The new dataset, dubbed XNLIeu, has been developed by first machine-translating the English XNLI corpus into Basque, followed by a manual post-edition step. We have conducted a series of experiments using mono- and multilingual LLMs to assess a) the effect of professional post-edition on the MT system; b) the best cross-lingual strategy for NLI in Basque; and c) whether the choice of the best cross-lingual strategy is influenced by the fact that the dataset is built by translation. The results show that post-edition is necessary and that the translate-train cross-lingual strategy obtains better results overall, although the gain is lower when tested in a dataset that has been built natively from scratch. Our code and datasets are publicly available under open licenses. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_06996 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | XNLIeu: a dataset for cross-lingual NLI in Basque Heredia, Maite Etxaniz, Julen Zulaika, Muitze Saralegi, Xabier Barnes, Jeremy Soroa, Aitor Computation and Language Artificial Intelligence XNLI is a popular Natural Language Inference (NLI) benchmark widely used to evaluate cross-lingual Natural Language Understanding (NLU) capabilities across languages. In this paper, we expand XNLI to include Basque, a low-resource language that can greatly benefit from transfer-learning approaches. The new dataset, dubbed XNLIeu, has been developed by first machine-translating the English XNLI corpus into Basque, followed by a manual post-edition step. We have conducted a series of experiments using mono- and multilingual LLMs to assess a) the effect of professional post-edition on the MT system; b) the best cross-lingual strategy for NLI in Basque; and c) whether the choice of the best cross-lingual strategy is influenced by the fact that the dataset is built by translation. The results show that post-edition is necessary and that the translate-train cross-lingual strategy obtains better results overall, although the gain is lower when tested in a dataset that has been built natively from scratch. Our code and datasets are publicly available under open licenses. |
| title | XNLIeu: a dataset for cross-lingual NLI in Basque |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2404.06996 |