MILPaC: A Novel Benchmark for Evaluating Translation of Legal Text to Indian Languages
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909379524034560 |
|---|---|
| author | Mahapatra, Sayan Datta, Debtanu Soni, Shubham Goswami, Adrijit Ghosh, Saptarshi |
| author_facet | Mahapatra, Sayan Datta, Debtanu Soni, Shubham Goswami, Adrijit Ghosh, Saptarshi |
| contents | Most legal text in the Indian judiciary is written in complex English due to historical reasons. However, only a small fraction of the Indian population is comfortable in reading English. Hence legal text needs to be made available in various Indian languages, possibly by translating the available legal text from English. Though there has been a lot of research on translation to and between Indian languages, to our knowledge, there has not been much prior work on such translation in the legal domain. In this work, we construct the first high-quality legal parallel corpus containing aligned text units in English and nine Indian languages, that includes several low-resource languages. We also benchmark the performance of a wide variety of Machine Translation (MT) systems over this corpus, including commercial MT systems, open-source MT systems and Large Language Models. Through a comprehensive survey by Law practitioners, we check how satisfied they are with the translations by some of these MT systems, and how well automatic MT evaluation metrics agree with the opinions of Law practitioners. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_09765 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | MILPaC: A Novel Benchmark for Evaluating Translation of Legal Text to Indian Languages Mahapatra, Sayan Datta, Debtanu Soni, Shubham Goswami, Adrijit Ghosh, Saptarshi Computation and Language Artificial Intelligence Most legal text in the Indian judiciary is written in complex English due to historical reasons. However, only a small fraction of the Indian population is comfortable in reading English. Hence legal text needs to be made available in various Indian languages, possibly by translating the available legal text from English. Though there has been a lot of research on translation to and between Indian languages, to our knowledge, there has not been much prior work on such translation in the legal domain. In this work, we construct the first high-quality legal parallel corpus containing aligned text units in English and nine Indian languages, that includes several low-resource languages. We also benchmark the performance of a wide variety of Machine Translation (MT) systems over this corpus, including commercial MT systems, open-source MT systems and Large Language Models. Through a comprehensive survey by Law practitioners, we check how satisfied they are with the translations by some of these MT systems, and how well automatic MT evaluation metrics agree with the opinions of Law practitioners. |
| title | MILPaC: A Novel Benchmark for Evaluating Translation of Legal Text to Indian Languages |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2310.09765 |