Walia-LLM: Enhancing Amharic-LLaMA by Integrating Task-Specific and Generative Datasets
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913332839055360 |
|---|---|
| author | Azime, Israel Abebe Tonja, Atnafu Lambebo Belay, Tadesse Destaw Fuge, Mitiku Yohannes Wassie, Aman Kassahun Jada, Eyasu Shiferaw Chanie, Yonas Sewunetie, Walelign Tewabe Yimam, Seid Muhie |
| author_facet | Azime, Israel Abebe Tonja, Atnafu Lambebo Belay, Tadesse Destaw Fuge, Mitiku Yohannes Wassie, Aman Kassahun Jada, Eyasu Shiferaw Chanie, Yonas Sewunetie, Walelign Tewabe Yimam, Seid Muhie |
| contents | Large language models (LLMs) have received a lot of attention in natural language processing (NLP) research because of their exceptional performance in understanding and generating human languages. However, low-resource languages are left behind due to the unavailability of resources. In this work, we focus on enhancing the LLaMA-2-Amharic model by integrating task-specific and generative datasets to improve language model performance for Amharic. We compile an Amharic instruction fine-tuning dataset and fine-tuned LLaMA-2-Amharic model. The fine-tuned model shows promising results in different NLP tasks. We open-source our dataset creation pipeline, instruction datasets, trained models, and evaluation outputs to promote language-specific studies on these models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_08015 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Walia-LLM: Enhancing Amharic-LLaMA by Integrating Task-Specific and Generative Datasets Azime, Israel Abebe Tonja, Atnafu Lambebo Belay, Tadesse Destaw Fuge, Mitiku Yohannes Wassie, Aman Kassahun Jada, Eyasu Shiferaw Chanie, Yonas Sewunetie, Walelign Tewabe Yimam, Seid Muhie Computation and Language Large language models (LLMs) have received a lot of attention in natural language processing (NLP) research because of their exceptional performance in understanding and generating human languages. However, low-resource languages are left behind due to the unavailability of resources. In this work, we focus on enhancing the LLaMA-2-Amharic model by integrating task-specific and generative datasets to improve language model performance for Amharic. We compile an Amharic instruction fine-tuning dataset and fine-tuned LLaMA-2-Amharic model. The fine-tuned model shows promising results in different NLP tasks. We open-source our dataset creation pipeline, instruction datasets, trained models, and evaluation outputs to promote language-specific studies on these models. |
| title | Walia-LLM: Enhancing Amharic-LLaMA by Integrating Task-Specific and Generative Datasets |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2402.08015 |