Native vs Non-Native Language Prompting: A Comparative Analysis
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914965166751744 |
|---|---|
| author | Kmainasi, Mohamed Bayan Khan, Rakif Shahroor, Ali Ezzat Bendou, Boushra Hasanain, Maram Alam, Firoj |
| author_facet | Kmainasi, Mohamed Bayan Khan, Rakif Shahroor, Ali Ezzat Bendou, Boushra Hasanain, Maram Alam, Firoj |
| contents | Large language models (LLMs) have shown remarkable abilities in different fields, including standard Natural Language Processing (NLP) tasks. To elicit knowledge from LLMs, prompts play a key role, consisting of natural language instructions. Most open and closed source LLMs are trained on available labeled and unlabeled resources--digital content such as text, images, audio, and videos. Hence, these models have better knowledge for high-resourced languages but struggle with low-resourced languages. Since prompts play a crucial role in understanding their capabilities, the language used for prompts remains an important research question. Although there has been significant research in this area, it is still limited, and less has been explored for medium to low-resourced languages. In this study, we investigate different prompting strategies (native vs. non-native) on 11 different NLP tasks associated with 12 different Arabic datasets (9.7K data points). In total, we conducted 197 experiments involving 3 LLMs, 12 datasets, and 3 prompting strategies. Our findings suggest that, on average, the non-native prompt performs the best, followed by mixed and native prompts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_07054 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Native vs Non-Native Language Prompting: A Comparative Analysis Kmainasi, Mohamed Bayan Khan, Rakif Shahroor, Ali Ezzat Bendou, Boushra Hasanain, Maram Alam, Firoj Computation and Language Artificial Intelligence 68T50 F.2.2; I.2.7 Large language models (LLMs) have shown remarkable abilities in different fields, including standard Natural Language Processing (NLP) tasks. To elicit knowledge from LLMs, prompts play a key role, consisting of natural language instructions. Most open and closed source LLMs are trained on available labeled and unlabeled resources--digital content such as text, images, audio, and videos. Hence, these models have better knowledge for high-resourced languages but struggle with low-resourced languages. Since prompts play a crucial role in understanding their capabilities, the language used for prompts remains an important research question. Although there has been significant research in this area, it is still limited, and less has been explored for medium to low-resourced languages. In this study, we investigate different prompting strategies (native vs. non-native) on 11 different NLP tasks associated with 12 different Arabic datasets (9.7K data points). In total, we conducted 197 experiments involving 3 LLMs, 12 datasets, and 3 prompting strategies. Our findings suggest that, on average, the non-native prompt performs the best, followed by mixed and native prompts. |
| title | Native vs Non-Native Language Prompting: A Comparative Analysis |
| topic | Computation and Language Artificial Intelligence 68T50 F.2.2; I.2.7 |
| url | https://arxiv.org/abs/2409.07054 |