3LM: Bridging Arabic, STEM, and Code through Benchmarking
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915408847568896 |
|---|---|
| author | Boussaha, Basma El Amel AlQadi, Leen Farooq, Mugariya Alsuwaidi, Shaikha Campesan, Giulia Alzubaidi, Ahmed Alyafeai, Mohammed Hacid, Hakim |
| author_facet | Boussaha, Basma El Amel AlQadi, Leen Farooq, Mugariya Alsuwaidi, Shaikha Campesan, Giulia Alzubaidi, Ahmed Alyafeai, Mohammed Hacid, Hakim |
| contents | Arabic is one of the most widely spoken languages in the world, yet efforts to develop and evaluate Large Language Models (LLMs) for Arabic remain relatively limited. Most existing Arabic benchmarks focus on linguistic, cultural, or religious content, leaving a significant gap in domains like STEM and code which are increasingly relevant for real-world LLM applications. To help bridge this gap, we present 3LM, a suite of three benchmarks designed specifically for Arabic. The first is a set of STEM-related question-answer pairs, naturally sourced from Arabic textbooks and educational worksheets. The second consists of synthetically generated STEM questions, created using the same sources. The third benchmark focuses on code generation, built through a careful translation of two widely used code benchmarks, incorporating a human-in-the-loop process with several rounds of review to ensure high-quality and faithful translations. We release all three benchmarks publicly to support the growth of Arabic LLM research in these essential but underrepresented areas. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_15850 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | 3LM: Bridging Arabic, STEM, and Code through Benchmarking Boussaha, Basma El Amel AlQadi, Leen Farooq, Mugariya Alsuwaidi, Shaikha Campesan, Giulia Alzubaidi, Ahmed Alyafeai, Mohammed Hacid, Hakim Computation and Language Arabic is one of the most widely spoken languages in the world, yet efforts to develop and evaluate Large Language Models (LLMs) for Arabic remain relatively limited. Most existing Arabic benchmarks focus on linguistic, cultural, or religious content, leaving a significant gap in domains like STEM and code which are increasingly relevant for real-world LLM applications. To help bridge this gap, we present 3LM, a suite of three benchmarks designed specifically for Arabic. The first is a set of STEM-related question-answer pairs, naturally sourced from Arabic textbooks and educational worksheets. The second consists of synthetically generated STEM questions, created using the same sources. The third benchmark focuses on code generation, built through a careful translation of two widely used code benchmarks, incorporating a human-in-the-loop process with several rounds of review to ensure high-quality and faithful translations. We release all three benchmarks publicly to support the growth of Arabic LLM research in these essential but underrepresented areas. |
| title | 3LM: Bridging Arabic, STEM, and Code through Benchmarking |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2507.15850 |