Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.20557 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917348141694976 |
|---|---|
| author | Liu, Hannah Min, Junghyun Lee, En-Shiun Annie Cheung, Ethan Yue Heng Hung, Shou-Yi Chan, Elsie Qian, Shiyao Liang, Runtong Huynh, Kimlan Yip, Wing Yu Ng, York Hay Yau, TSZ Fung Lo, Ka Ieng Charlotte Wu, You-Wei Tsai, Richard Tzong-Han |
| author_facet | Liu, Hannah Min, Junghyun Lee, En-Shiun Annie Cheung, Ethan Yue Heng Hung, Shou-Yi Chan, Elsie Qian, Shiyao Liang, Runtong Huynh, Kimlan Yip, Wing Yu Ng, York Hay Yau, TSZ Fung Lo, Ka Ieng Charlotte Wu, You-Wei Tsai, Richard Tzong-Han |
| contents | Despite major advances in machine translation (MT) in recent years, progress remains limited for many low-resource languages that lack large-scale training data and linguistic resources. In this paper, we introduce \dsname, a novel fine-grained dataset that builds on existing parallel corpora to provide error span, error type, and error severity annotations in machine-translated examples from English to Mandarin, Cantonese, and Wu Chinese, along with a Mandarin-Hokkien component derived from a non-parallel source. Our dataset serves as a resource for the MT community to fine-tune models with error detection capabilities, supporting research on translation quality estimation, error-aware generation, and low-resource language evaluation. We also establish baseline results using language models to benchmark translation error detection performance. Specifically, we evaluate multiple open source and closed source LLMs using span-level and correlation-based MQM metrics, revealing their limited precision, underscoring the need for our dataset. Finally, we report our rigorous annotation process by native speakers, with analyses on pilot studies, iterative feedback, insights, and patterns in error type and severity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_20557 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SiniticMTError: A Machine Translation Dataset with Error Annotations for Sinitic Languages Liu, Hannah Min, Junghyun Lee, En-Shiun Annie Cheung, Ethan Yue Heng Hung, Shou-Yi Chan, Elsie Qian, Shiyao Liang, Runtong Huynh, Kimlan Yip, Wing Yu Ng, York Hay Yau, TSZ Fung Lo, Ka Ieng Charlotte Wu, You-Wei Tsai, Richard Tzong-Han Computation and Language Despite major advances in machine translation (MT) in recent years, progress remains limited for many low-resource languages that lack large-scale training data and linguistic resources. In this paper, we introduce \dsname, a novel fine-grained dataset that builds on existing parallel corpora to provide error span, error type, and error severity annotations in machine-translated examples from English to Mandarin, Cantonese, and Wu Chinese, along with a Mandarin-Hokkien component derived from a non-parallel source. Our dataset serves as a resource for the MT community to fine-tune models with error detection capabilities, supporting research on translation quality estimation, error-aware generation, and low-resource language evaluation. We also establish baseline results using language models to benchmark translation error detection performance. Specifically, we evaluate multiple open source and closed source LLMs using span-level and correlation-based MQM metrics, revealing their limited precision, underscoring the need for our dataset. Finally, we report our rigorous annotation process by native speakers, with analyses on pilot studies, iterative feedback, insights, and patterns in error type and severity. |
| title | SiniticMTError: A Machine Translation Dataset with Error Annotations for Sinitic Languages |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.20557 |