Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.20451 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916712416280576 |
|---|---|
| author | Lee, Daniel Sharma, Harsh Han, Jieun Jeong, Sunny Oh, Alice Shwartz, Vered |
| author_facet | Lee, Daniel Sharma, Harsh Han, Jieun Jeong, Sunny Oh, Alice Shwartz, Vered |
| contents | Translating knowledge-intensive and entity-rich text between English and Korean requires transcreation to preserve language-specific and cultural nuances beyond literal, phonetic or word-for-word conversion. We evaluate 13 models (LLMs and MT models) using automatic metrics and human assessment by bilingual annotators. Our findings show LLMs outperform traditional MT systems but struggle with entity translation requiring cultural adaptation. By constructing an error taxonomy, we identify incorrect responses and entity name errors as key issues, with performance varying by entity type and popularity level. This work exposes gaps in automatic evaluation metrics and hope to enable future work in completing culturally-nuanced machine translation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_20451 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Team ACK at SemEval-2025 Task 2: Beyond Word-for-Word Machine Translation for English-Korean Pairs Lee, Daniel Sharma, Harsh Han, Jieun Jeong, Sunny Oh, Alice Shwartz, Vered Computation and Language Information Retrieval Translating knowledge-intensive and entity-rich text between English and Korean requires transcreation to preserve language-specific and cultural nuances beyond literal, phonetic or word-for-word conversion. We evaluate 13 models (LLMs and MT models) using automatic metrics and human assessment by bilingual annotators. Our findings show LLMs outperform traditional MT systems but struggle with entity translation requiring cultural adaptation. By constructing an error taxonomy, we identify incorrect responses and entity name errors as key issues, with performance varying by entity type and popularity level. This work exposes gaps in automatic evaluation metrics and hope to enable future work in completing culturally-nuanced machine translation. |
| title | Team ACK at SemEval-2025 Task 2: Beyond Word-for-Word Machine Translation for English-Korean Pairs |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2504.20451 |