Align-then-Slide: A complete evaluation framework for Ultra-Long Document-Level Machine Translation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908518928351232 |
|---|---|
| author | Guo, Jiaxin Wei, Daimeng Luo, Yuanchang Chen, Xiaoyu Wu, Zhanglin Yang, Huan Shang, Hengchao Li, Zongyao Rao, Zhiqiang Yang, Jinlong Yang, Hao |
| author_facet | Guo, Jiaxin Wei, Daimeng Luo, Yuanchang Chen, Xiaoyu Wu, Zhanglin Yang, Huan Shang, Hengchao Li, Zongyao Rao, Zhiqiang Yang, Jinlong Yang, Hao |
| contents | Large language models (LLMs) have ushered in a new era for document-level machine translation (\textit{doc}-mt), yet their whole-document outputs challenge existing evaluation methods that assume sentence-by-sentence alignment. We introduce \textit{\textbf{Align-then-Slide}}, a complete evaluation framework for ultra-long doc-mt. In the Align stage, we automatically infer sentence-level source-target correspondences and rebuild the target to match the source sentence number, resolving omissions and many-to-one/one-to-many mappings. In the n-Chunk Sliding Evaluate stage, we calculate averaged metric scores under 1-, 2-, 3- and 4-chunk for multi-granularity assessment. Experiments on the WMT benchmark show a Pearson correlation of 0.929 between our method with expert MQM rankings. On a newly curated real-world test set, our method again aligns closely with human judgments. Furthermore, preference data produced by Align-then-Slide enables effective CPO training and its direct use as a reward model for GRPO, both yielding translations preferred over a vanilla SFT baseline. The results validate our framework as an accurate, robust, and actionable evaluation tool for doc-mt systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_03809 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Align-then-Slide: A complete evaluation framework for Ultra-Long Document-Level Machine Translation Guo, Jiaxin Wei, Daimeng Luo, Yuanchang Chen, Xiaoyu Wu, Zhanglin Yang, Huan Shang, Hengchao Li, Zongyao Rao, Zhiqiang Yang, Jinlong Yang, Hao Computation and Language Artificial Intelligence Large language models (LLMs) have ushered in a new era for document-level machine translation (\textit{doc}-mt), yet their whole-document outputs challenge existing evaluation methods that assume sentence-by-sentence alignment. We introduce \textit{\textbf{Align-then-Slide}}, a complete evaluation framework for ultra-long doc-mt. In the Align stage, we automatically infer sentence-level source-target correspondences and rebuild the target to match the source sentence number, resolving omissions and many-to-one/one-to-many mappings. In the n-Chunk Sliding Evaluate stage, we calculate averaged metric scores under 1-, 2-, 3- and 4-chunk for multi-granularity assessment. Experiments on the WMT benchmark show a Pearson correlation of 0.929 between our method with expert MQM rankings. On a newly curated real-world test set, our method again aligns closely with human judgments. Furthermore, preference data produced by Align-then-Slide enables effective CPO training and its direct use as a reward model for GRPO, both yielding translations preferred over a vanilla SFT baseline. The results validate our framework as an accurate, robust, and actionable evaluation tool for doc-mt systems. |
| title | Align-then-Slide: A complete evaluation framework for Ultra-Long Document-Level Machine Translation |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2509.03809 |