Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.22603 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915417691258880 |
|---|---|
| author | Al-Matham, Rawan Darwish, Kareem Al-Rasheed, Raghad Alshammari, Waad Alhoshan, Muneera Almazrua, Amal Wazrah, Asma Al Alheraki, Mais Alam, Firoj Nakov, Preslav Alzahrani, Norah alBilali, Eman Habash, Nizar El-Sheikh, Abdelrahman Elmallah, Muhammad Li, Haonan Mubarak, Hamdy Anwar, Mohamed Alyafeai, Zaid Abdelali, Ahmed Altwairesh, Nora Hasanain, Maram Thubaity, Abdulmohsen Al Shehata, Shady Alhafni, Bashar Hamed, Injy Inoue, Go Elmadani, Khalid Obeid, Ossama Haouari, Fatima Elsayed, Tamer Alghamdi, Emad Almubarak, Khalid Alshahrani, Saied Aljarrah, Ola Alajlan, Safa Alshaqarawi, Areej Alshihri, Maryam Alghurabi, Sultana Alzeghayer, Atikah Altamimi, Afrah Alfaifi, Abdullah AlOsaimy, Abdulrahman |
| author_facet | Al-Matham, Rawan Darwish, Kareem Al-Rasheed, Raghad Alshammari, Waad Alhoshan, Muneera Almazrua, Amal Wazrah, Asma Al Alheraki, Mais Alam, Firoj Nakov, Preslav Alzahrani, Norah alBilali, Eman Habash, Nizar El-Sheikh, Abdelrahman Elmallah, Muhammad Li, Haonan Mubarak, Hamdy Anwar, Mohamed Alyafeai, Zaid Abdelali, Ahmed Altwairesh, Nora Hasanain, Maram Thubaity, Abdulmohsen Al Shehata, Shady Alhafni, Bashar Hamed, Injy Inoue, Go Elmadani, Khalid Obeid, Ossama Haouari, Fatima Elsayed, Tamer Alghamdi, Emad Almubarak, Khalid Alshahrani, Saied Aljarrah, Ola Alajlan, Safa Alshaqarawi, Areej Alshihri, Maryam Alghurabi, Sultana Alzeghayer, Atikah Altamimi, Afrah Alfaifi, Abdullah AlOsaimy, Abdulrahman |
| contents | The impressive advancement of Large Language Models (LLMs) in English has not been matched across all languages. In particular, LLM performance in Arabic lags behind, due to data scarcity, linguistic diversity of Arabic and its dialects, morphological complexity, etc. Progress is further hindered by the quality of Arabic benchmarks, which typically rely on static, publicly available data, lack comprehensive task coverage, or do not provide dedicated platforms with blind test sets. This makes it challenging to measure actual progress and to mitigate data contamination. Here, we aim to bridge these gaps. In particular, we introduce BALSAM, a comprehensive, community-driven benchmark aimed at advancing Arabic LLM development and evaluation. It includes 78 NLP tasks from 14 broad categories, with 52K examples divided into 37K test and 15K development, and a centralized, transparent platform for blind evaluation. We envision BALSAM as a unifying platform that sets standards and promotes collaborative research to advance Arabic LLM capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_22603 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | BALSAM: A Platform for Benchmarking Arabic Large Language Models Al-Matham, Rawan Darwish, Kareem Al-Rasheed, Raghad Alshammari, Waad Alhoshan, Muneera Almazrua, Amal Wazrah, Asma Al Alheraki, Mais Alam, Firoj Nakov, Preslav Alzahrani, Norah alBilali, Eman Habash, Nizar El-Sheikh, Abdelrahman Elmallah, Muhammad Li, Haonan Mubarak, Hamdy Anwar, Mohamed Alyafeai, Zaid Abdelali, Ahmed Altwairesh, Nora Hasanain, Maram Thubaity, Abdulmohsen Al Shehata, Shady Alhafni, Bashar Hamed, Injy Inoue, Go Elmadani, Khalid Obeid, Ossama Haouari, Fatima Elsayed, Tamer Alghamdi, Emad Almubarak, Khalid Alshahrani, Saied Aljarrah, Ola Alajlan, Safa Alshaqarawi, Areej Alshihri, Maryam Alghurabi, Sultana Alzeghayer, Atikah Altamimi, Afrah Alfaifi, Abdullah AlOsaimy, Abdulrahman Computation and Language Artificial Intelligence The impressive advancement of Large Language Models (LLMs) in English has not been matched across all languages. In particular, LLM performance in Arabic lags behind, due to data scarcity, linguistic diversity of Arabic and its dialects, morphological complexity, etc. Progress is further hindered by the quality of Arabic benchmarks, which typically rely on static, publicly available data, lack comprehensive task coverage, or do not provide dedicated platforms with blind test sets. This makes it challenging to measure actual progress and to mitigate data contamination. Here, we aim to bridge these gaps. In particular, we introduce BALSAM, a comprehensive, community-driven benchmark aimed at advancing Arabic LLM development and evaluation. It includes 78 NLP tasks from 14 broad categories, with 52K examples divided into 37K test and 15K development, and a centralized, transparent platform for blind evaluation. We envision BALSAM as a unifying platform that sets standards and promotes collaborative research to advance Arabic LLM capabilities. |
| title | BALSAM: A Platform for Benchmarking Arabic Large Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2507.22603 |