Evaluating the Capabilities of Large Language Models for Multi-label Emotion Understanding
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929656896159744 |
|---|---|
| author | Belay, Tadesse Destaw Azime, Israel Abebe Ayele, Abinew Ali Sidorov, Grigori Klakow, Dietrich Slusallek, Philipp Kolesnikova, Olga Yimam, Seid Muhie |
| author_facet | Belay, Tadesse Destaw Azime, Israel Abebe Ayele, Abinew Ali Sidorov, Grigori Klakow, Dietrich Slusallek, Philipp Kolesnikova, Olga Yimam, Seid Muhie |
| contents | Large Language Models (LLMs) show promising learning and reasoning abilities. Compared to other NLP tasks, multilingual and multi-label emotion evaluation tasks are under-explored in LLMs. In this paper, we present EthioEmo, a multi-label emotion classification dataset for four Ethiopian languages, namely, Amharic (amh), Afan Oromo (orm), Somali (som), and Tigrinya (tir). We perform extensive experiments with an additional English multi-label emotion dataset from SemEval 2018 Task 1. Our evaluation includes encoder-only, encoder-decoder, and decoder-only language models. We compare zero and few-shot approaches of LLMs to fine-tuning smaller language models. The results show that accurate multi-label emotion classification is still insufficient even for high-resource languages such as English, and there is a large gap between the performance of high-resource and low-resource languages. The results also show varying performance levels depending on the language and model type. EthioEmo is available publicly to further improve the understanding of emotions in language models and how people convey emotions through various languages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_17837 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Evaluating the Capabilities of Large Language Models for Multi-label Emotion Understanding Belay, Tadesse Destaw Azime, Israel Abebe Ayele, Abinew Ali Sidorov, Grigori Klakow, Dietrich Slusallek, Philipp Kolesnikova, Olga Yimam, Seid Muhie Computation and Language Large Language Models (LLMs) show promising learning and reasoning abilities. Compared to other NLP tasks, multilingual and multi-label emotion evaluation tasks are under-explored in LLMs. In this paper, we present EthioEmo, a multi-label emotion classification dataset for four Ethiopian languages, namely, Amharic (amh), Afan Oromo (orm), Somali (som), and Tigrinya (tir). We perform extensive experiments with an additional English multi-label emotion dataset from SemEval 2018 Task 1. Our evaluation includes encoder-only, encoder-decoder, and decoder-only language models. We compare zero and few-shot approaches of LLMs to fine-tuning smaller language models. The results show that accurate multi-label emotion classification is still insufficient even for high-resource languages such as English, and there is a large gap between the performance of high-resource and low-resource languages. The results also show varying performance levels depending on the language and model type. EthioEmo is available publicly to further improve the understanding of emotions in language models and how people convey emotions through various languages. |
| title | Evaluating the Capabilities of Large Language Models for Multi-label Emotion Understanding |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2412.17837 |