Are Generative Language Models Multicultural? A Study on Hausa Culture and Emotions using ChatGPT
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914850916007936 |
|---|---|
| author | Ahmad, Ibrahim Said Dudy, Shiran Ramachandranpillai, Resmi Church, Kenneth |
| author_facet | Ahmad, Ibrahim Said Dudy, Shiran Ramachandranpillai, Resmi Church, Kenneth |
| contents | Large Language Models (LLMs), such as ChatGPT, are widely used to generate content for various purposes and audiences. However, these models may not reflect the cultural and emotional diversity of their users, especially for low-resource languages. In this paper, we investigate how ChatGPT represents Hausa's culture and emotions. We compare responses generated by ChatGPT with those provided by native Hausa speakers on 37 culturally relevant questions. We conducted experiments using emotion analysis and applied two similarity metrics to measure the alignment between human and ChatGPT responses. We also collected human participants ratings and feedback on ChatGPT responses. Our results show that ChatGPT has some level of similarity to human responses, but also exhibits some gaps and biases in its knowledge and awareness of the Hausa culture and emotions. We discuss the implications and limitations of our methodology and analysis and suggest ways to improve the performance and evaluation of LLMs for low-resource languages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_19504 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Are Generative Language Models Multicultural? A Study on Hausa Culture and Emotions using ChatGPT Ahmad, Ibrahim Said Dudy, Shiran Ramachandranpillai, Resmi Church, Kenneth Computation and Language Large Language Models (LLMs), such as ChatGPT, are widely used to generate content for various purposes and audiences. However, these models may not reflect the cultural and emotional diversity of their users, especially for low-resource languages. In this paper, we investigate how ChatGPT represents Hausa's culture and emotions. We compare responses generated by ChatGPT with those provided by native Hausa speakers on 37 culturally relevant questions. We conducted experiments using emotion analysis and applied two similarity metrics to measure the alignment between human and ChatGPT responses. We also collected human participants ratings and feedback on ChatGPT responses. Our results show that ChatGPT has some level of similarity to human responses, but also exhibits some gaps and biases in its knowledge and awareness of the Hausa culture and emotions. We discuss the implications and limitations of our methodology and analysis and suggest ways to improve the performance and evaluation of LLMs for low-resource languages. |
| title | Are Generative Language Models Multicultural? A Study on Hausa Culture and Emotions using ChatGPT |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2406.19504 |