XIFBench: Evaluating Large Language Models on Multilingual Instruction Following
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909881278136320 |
|---|---|
| author | Li, Zhenyu Chen, Kehai Long, Yunfei Bai, Xuefeng Zhang, Yaoyin Wei, Xuchen Li, Juntao Zhang, Min |
| author_facet | Li, Zhenyu Chen, Kehai Long, Yunfei Bai, Xuefeng Zhang, Yaoyin Wei, Xuchen Li, Juntao Zhang, Min |
| contents | Large Language Models (LLMs) have demonstrated remarkable instruction-following capabilities across various applications. However, their performance in multilingual settings lacks systematic investigation, with existing evaluations lacking fine-grained constraint analysis across diverse linguistic contexts. We introduce XIFBench, a comprehensive constraint-based benchmark for evaluating multilingual instruction-following abilities of LLMs, comprising 558 instructions with 0-5 additional constraints across five categories (Content, Style, Situation, Format, and Numerical) in six languages spanning different resource levels. To support reliable and consistent cross-lingual evaluation, we implement three methodological innovations: cultural accessibility annotation, constraint-level translation validation, and requirement-based evaluation using English requirements as semantic anchors across languages. Extensive experiments with various LLMs not only quantify performance disparities across resource levels but also provide detailed insights into how language resources, constraint categories, instruction complexity, and cultural specificity influence multilingual instruction-following. Our code and data are available at https://github.com/zhenyuli801/XIFBench. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_07539 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | XIFBench: Evaluating Large Language Models on Multilingual Instruction Following Li, Zhenyu Chen, Kehai Long, Yunfei Bai, Xuefeng Zhang, Yaoyin Wei, Xuchen Li, Juntao Zhang, Min Computation and Language Large Language Models (LLMs) have demonstrated remarkable instruction-following capabilities across various applications. However, their performance in multilingual settings lacks systematic investigation, with existing evaluations lacking fine-grained constraint analysis across diverse linguistic contexts. We introduce XIFBench, a comprehensive constraint-based benchmark for evaluating multilingual instruction-following abilities of LLMs, comprising 558 instructions with 0-5 additional constraints across five categories (Content, Style, Situation, Format, and Numerical) in six languages spanning different resource levels. To support reliable and consistent cross-lingual evaluation, we implement three methodological innovations: cultural accessibility annotation, constraint-level translation validation, and requirement-based evaluation using English requirements as semantic anchors across languages. Extensive experiments with various LLMs not only quantify performance disparities across resource levels but also provide detailed insights into how language resources, constraint categories, instruction complexity, and cultural specificity influence multilingual instruction-following. Our code and data are available at https://github.com/zhenyuli801/XIFBench. |
| title | XIFBench: Evaluating Large Language Models on Multilingual Instruction Following |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.07539 |