CultranAI at PalmX 2025: Data Augmentation for Cultural Knowledge Representation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918152067088384 |
|---|---|
| author | Bhatti, Hunzalah Hassan Ahmed, Youssef Hasan, Md Arid Alam, Firoj |
| author_facet | Bhatti, Hunzalah Hassan Ahmed, Youssef Hasan, Md Arid Alam, Firoj |
| contents | In this paper, we report our participation to the PalmX cultural evaluation shared task. Our system, CultranAI, focused on data augmentation and LoRA fine-tuning of large language models (LLMs) for Arabic cultural knowledge representation. We benchmarked several LLMs to identify the best-performing model for the task. In addition to utilizing the PalmX dataset, we augmented it by incorporating the Palm dataset and curated a new dataset of over 22K culturally grounded multiple-choice questions (MCQs). Our experiments showed that the Fanar-1-9B-Instruct model achieved the highest performance. We fine-tuned this model on the combined augmented dataset of 22K+ MCQs. On the blind test set, our submitted system ranked 5th with an accuracy of 70.50%, while on the PalmX development set, it achieved an accuracy of 84.1%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_17324 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CultranAI at PalmX 2025: Data Augmentation for Cultural Knowledge Representation Bhatti, Hunzalah Hassan Ahmed, Youssef Hasan, Md Arid Alam, Firoj Computation and Language Artificial Intelligence 68T50 F.2.2; I.2.7 In this paper, we report our participation to the PalmX cultural evaluation shared task. Our system, CultranAI, focused on data augmentation and LoRA fine-tuning of large language models (LLMs) for Arabic cultural knowledge representation. We benchmarked several LLMs to identify the best-performing model for the task. In addition to utilizing the PalmX dataset, we augmented it by incorporating the Palm dataset and curated a new dataset of over 22K culturally grounded multiple-choice questions (MCQs). Our experiments showed that the Fanar-1-9B-Instruct model achieved the highest performance. We fine-tuned this model on the combined augmented dataset of 22K+ MCQs. On the blind test set, our submitted system ranked 5th with an accuracy of 70.50%, while on the PalmX development set, it achieved an accuracy of 84.1%. |
| title | CultranAI at PalmX 2025: Data Augmentation for Cultural Knowledge Representation |
| topic | Computation and Language Artificial Intelligence 68T50 F.2.2; I.2.7 |
| url | https://arxiv.org/abs/2508.17324 |