Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914680886263808 |
|---|---|
| author | Wang, Wenxuan Jiao, Wenxiang Huang, Jingyuan Dai, Ruyi Huang, Jen-tse Tu, Zhaopeng Lyu, Michael R. |
| author_facet | Wang, Wenxuan Jiao, Wenxiang Huang, Jingyuan Dai, Ruyi Huang, Jen-tse Tu, Zhaopeng Lyu, Michael R. |
| contents | This paper identifies a cultural dominance issue within large language models (LLMs) due to the predominant use of English data in model training (e.g., ChatGPT). LLMs often provide inappropriate English-culture-related answers that are not relevant to the expected culture when users ask in non-English languages. To systematically evaluate the cultural dominance issue, we build a benchmark of concrete (e.g., holidays and songs) and abstract (e.g., values and opinions) cultural objects. Empirical results show that the representative GPT models suffer from the culture dominance problem, where GPT-4 is the most affected while text-davinci-003 suffers the least from this problem. Our study emphasizes the need to critically examine cultural dominance and ethical consideration in their development and deployment. We show that two straightforward methods in model development (i.e., pretraining on more diverse data) and deployment (e.g., culture-aware prompting) can significantly mitigate the cultural dominance issue in LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_12481 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models Wang, Wenxuan Jiao, Wenxiang Huang, Jingyuan Dai, Ruyi Huang, Jen-tse Tu, Zhaopeng Lyu, Michael R. Computation and Language Artificial Intelligence This paper identifies a cultural dominance issue within large language models (LLMs) due to the predominant use of English data in model training (e.g., ChatGPT). LLMs often provide inappropriate English-culture-related answers that are not relevant to the expected culture when users ask in non-English languages. To systematically evaluate the cultural dominance issue, we build a benchmark of concrete (e.g., holidays and songs) and abstract (e.g., values and opinions) cultural objects. Empirical results show that the representative GPT models suffer from the culture dominance problem, where GPT-4 is the most affected while text-davinci-003 suffers the least from this problem. Our study emphasizes the need to critically examine cultural dominance and ethical consideration in their development and deployment. We show that two straightforward methods in model development (i.e., pretraining on more diverse data) and deployment (e.g., culture-aware prompting) can significantly mitigate the cultural dominance issue in LLMs. |
| title | Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2310.12481 |