Good, but not always Fair: An Evaluation of Gender Bias for three commercial Machine Translation Systems
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916178508644352 |
|---|---|
| author | Piazzolla, Silvia Alma Savoldi, Beatrice Bentivogli, Luisa |
| author_facet | Piazzolla, Silvia Alma Savoldi, Beatrice Bentivogli, Luisa |
| contents | Machine Translation (MT) continues to make significant strides in quality and is increasingly adopted on a larger scale. Consequently, analyses have been redirected to more nuanced aspects, intricate phenomena, as well as potential risks that may arise from the widespread use of MT tools. Along this line, this paper offers a meticulous assessment of three commercial MT systems - Google Translate, DeepL, and Modern MT - with a specific focus on gender translation and bias. For three language pairs (English/Spanish, English/Italian, and English/French), we scrutinize the behavior of such systems at several levels of granularity and on a variety of naturally occurring gender phenomena in translation. Our study takes stock of the current state of online MT tools, by revealing significant discrepancies in the gender translation of the three systems, with each system displaying varying degrees of bias despite their overall translation quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_05882 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Good, but not always Fair: An Evaluation of Gender Bias for three commercial Machine Translation Systems Piazzolla, Silvia Alma Savoldi, Beatrice Bentivogli, Luisa Computation and Language Machine Translation (MT) continues to make significant strides in quality and is increasingly adopted on a larger scale. Consequently, analyses have been redirected to more nuanced aspects, intricate phenomena, as well as potential risks that may arise from the widespread use of MT tools. Along this line, this paper offers a meticulous assessment of three commercial MT systems - Google Translate, DeepL, and Modern MT - with a specific focus on gender translation and bias. For three language pairs (English/Spanish, English/Italian, and English/French), we scrutinize the behavior of such systems at several levels of granularity and on a variety of naturally occurring gender phenomena in translation. Our study takes stock of the current state of online MT tools, by revealing significant discrepancies in the gender translation of the three systems, with each system displaying varying degrees of bias despite their overall translation quality. |
| title | Good, but not always Fair: An Evaluation of Gender Bias for three commercial Machine Translation Systems |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2306.05882 |