Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2408.16390 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929693380313088 |
|---|---|
| author | Li, Yunmeng Suzuki, Jun Morishita, Makoto Abe, Kaori Inui, Kentaro |
| author_facet | Li, Yunmeng Suzuki, Jun Morishita, Makoto Abe, Kaori Inui, Kentaro |
| contents | The complexities of chats pose significant challenges for machine translation models. Recognizing the need for a precise evaluation metric to address the issues of chat translation, this study introduces Multidimensional Quality Metrics for Chat Translation (MQM-Chat). Through the experiments of five models using MQM-Chat, we observed that all models generated certain fundamental errors, while each of them has different shortcomings, such as omission, overly correcting ambiguous source content, and buzzword issues, resulting in the loss of stylized information. Our findings underscore the effectiveness of MQM-Chat in evaluating chat translation, emphasizing the importance of stylized content and dialogue consistency for future studies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_16390 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MQM-Chat: Multidimensional Quality Metrics for Chat Translation Li, Yunmeng Suzuki, Jun Morishita, Makoto Abe, Kaori Inui, Kentaro Computation and Language The complexities of chats pose significant challenges for machine translation models. Recognizing the need for a precise evaluation metric to address the issues of chat translation, this study introduces Multidimensional Quality Metrics for Chat Translation (MQM-Chat). Through the experiments of five models using MQM-Chat, we observed that all models generated certain fundamental errors, while each of them has different shortcomings, such as omission, overly correcting ambiguous source content, and buzzword issues, resulting in the loss of stylized information. Our findings underscore the effectiveness of MQM-Chat in evaluating chat translation, emphasizing the importance of stylized content and dialogue consistency for future studies. |
| title | MQM-Chat: Multidimensional Quality Metrics for Chat Translation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2408.16390 |