Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.11334 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911593383591936 |
|---|---|
| author | Gao, Jian Xuan, Richeng Kang, Zhaolu Liao, Dingshi Huang, Wenxin Huang, Zongmou Xu, Yangdi Qin, Bowen He, Zheqi Yang, Xi Li, Changjin Lin, Yonghua |
| author_facet | Gao, Jian Xuan, Richeng Kang, Zhaolu Liao, Dingshi Huang, Wenxin Huang, Zongmou Xu, Yangdi Qin, Bowen He, Zheqi Yang, Xi Li, Changjin Lin, Yonghua |
| contents | The rapid advancement of large language models (LLMs) has not been matched by their evaluation in low-resource languages, especially Southeast Asian languages like Lao. To fill this gap, we introduce \textbf{LaoBench}, the first large-scale, high-quality, and multidimensional benchmark for assessing LLM language understanding and reasoning in Lao. LaoBench contains \textbf{17,000+} expert-curated samples across three dimensions: culturally grounded knowledge application, curriculum-aligned K12 education, and bilingual translation among Lao, Chinese, and English. It includes open-source and held-out subsets, where the held-out portion enables secure black-box evaluation via a controlled service to improve fairness and data security. We construct LaoBench with a hybrid pipeline that combines expert authoring with agent-assisted verification, ensuring linguistic accuracy, cultural relevance, and educational validity. We evaluate diverse state-of-the-art open-source and closed-source LLMs, and find that even strong multilingual models lag behind human experts, particularly in culturally grounded reasoning and translation fidelity. We hope LaoBench will catalyze research on Lao and other underrepresented Southeast Asian languages for more inclusive multilingual evaluation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_11334 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LaoBench: A Large-Scale Multidimensional Lao Benchmark for Large Language Models Gao, Jian Xuan, Richeng Kang, Zhaolu Liao, Dingshi Huang, Wenxin Huang, Zongmou Xu, Yangdi Qin, Bowen He, Zheqi Yang, Xi Li, Changjin Lin, Yonghua Computation and Language The rapid advancement of large language models (LLMs) has not been matched by their evaluation in low-resource languages, especially Southeast Asian languages like Lao. To fill this gap, we introduce \textbf{LaoBench}, the first large-scale, high-quality, and multidimensional benchmark for assessing LLM language understanding and reasoning in Lao. LaoBench contains \textbf{17,000+} expert-curated samples across three dimensions: culturally grounded knowledge application, curriculum-aligned K12 education, and bilingual translation among Lao, Chinese, and English. It includes open-source and held-out subsets, where the held-out portion enables secure black-box evaluation via a controlled service to improve fairness and data security. We construct LaoBench with a hybrid pipeline that combines expert authoring with agent-assisted verification, ensuring linguistic accuracy, cultural relevance, and educational validity. We evaluate diverse state-of-the-art open-source and closed-source LLMs, and find that even strong multilingual models lag behind human experts, particularly in culturally grounded reasoning and translation fidelity. We hope LaoBench will catalyze research on Lao and other underrepresented Southeast Asian languages for more inclusive multilingual evaluation. |
| title | LaoBench: A Large-Scale Multidimensional Lao Benchmark for Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2511.11334 |