Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2404.17342 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912201631072256 |
|---|---|
| author | Lynn, Teresa Altakrori, Malik H. Magdy, Samar Mohamed Das, Rocktim Jyoti Lyu, Chenyang Nasr, Mohamed Samih, Younes Chirkunov, Kirill Aji, Alham Fikri Nakov, Preslav Godbole, Shantanu Roukos, Salim Florian, Radu Habash, Nizar |
| author_facet | Lynn, Teresa Altakrori, Malik H. Magdy, Samar Mohamed Das, Rocktim Jyoti Lyu, Chenyang Nasr, Mohamed Samih, Younes Chirkunov, Kirill Aji, Alham Fikri Nakov, Preslav Godbole, Shantanu Roukos, Salim Florian, Radu Habash, Nizar |
| contents | The rapid evolution of Natural Language Processing (NLP) has favoured major languages such as English, leaving a significant gap for many others due to limited resources. This is especially evident in the context of data annotation, a task whose importance cannot be underestimated, but which is time-consuming and costly. Thus, any dataset for resource-poor languages is precious, in particular when it is task-specific. Here, we explore the feasibility of repurposing an existing multilingual dataset for a new NLP task: we repurpose a subset of the BELEBELE dataset (Bandarkar et al., 2023), which was designed for multiple-choice question answering (MCQA), to enable the more practical task of extractive QA (EQA) in the style of machine reading comprehension. We present annotation guidelines and a parallel EQA dataset for English and Modern Standard Arabic (MSA). We also present QA evaluation results for several monolingual and cross-lingual QA pairs including English, MSA, and five Arabic dialects. We aim to help others adapt our approach for the remaining 120 BELEBELE language variants, many of which are deemed under-resourced. We also provide a thorough analysis and share insights to deepen understanding of the challenges and opportunities in NLP task reformulation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_17342 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | From Multiple-Choice to Extractive QA: A Case Study for English and Arabic Lynn, Teresa Altakrori, Malik H. Magdy, Samar Mohamed Das, Rocktim Jyoti Lyu, Chenyang Nasr, Mohamed Samih, Younes Chirkunov, Kirill Aji, Alham Fikri Nakov, Preslav Godbole, Shantanu Roukos, Salim Florian, Radu Habash, Nizar Computation and Language Artificial Intelligence The rapid evolution of Natural Language Processing (NLP) has favoured major languages such as English, leaving a significant gap for many others due to limited resources. This is especially evident in the context of data annotation, a task whose importance cannot be underestimated, but which is time-consuming and costly. Thus, any dataset for resource-poor languages is precious, in particular when it is task-specific. Here, we explore the feasibility of repurposing an existing multilingual dataset for a new NLP task: we repurpose a subset of the BELEBELE dataset (Bandarkar et al., 2023), which was designed for multiple-choice question answering (MCQA), to enable the more practical task of extractive QA (EQA) in the style of machine reading comprehension. We present annotation guidelines and a parallel EQA dataset for English and Modern Standard Arabic (MSA). We also present QA evaluation results for several monolingual and cross-lingual QA pairs including English, MSA, and five Arabic dialects. We aim to help others adapt our approach for the remaining 120 BELEBELE language variants, many of which are deemed under-resourced. We also provide a thorough analysis and share insights to deepen understanding of the challenges and opportunities in NLP task reformulation. |
| title | From Multiple-Choice to Extractive QA: A Case Study for English and Arabic |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2404.17342 |