From Facts to Folklore: Evaluating Large Language Models on Bengali Cultural Knowledge
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918166817406976 |
|---|---|
| author | Chowdhury, Nafis Haque, Moinul Ahmed, Anika Tasnim, Nazia Shihab, Md. Istiak Hossain Rahman, Sajjadur Sadeque, Farig |
| author_facet | Chowdhury, Nafis Haque, Moinul Ahmed, Anika Tasnim, Nazia Shihab, Md. Istiak Hossain Rahman, Sajjadur Sadeque, Farig |
| contents | Recent progress in NLP research has demonstrated remarkable capabilities of large language models (LLMs) across a wide range of tasks. While recent multilingual benchmarks have advanced cultural evaluation for LLMs, critical gaps remain in capturing the nuances of low-resource cultures. Our work addresses these limitations through a Bengali Language Cultural Knowledge (BLanCK) dataset including folk traditions, culinary arts, and regional dialects. Our investigation of several multilingual language models shows that while these models perform well in non-cultural categories, they struggle significantly with cultural knowledge and performance improves substantially across all models when context is provided, emphasizing context-aware architectures and culturally curated training data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_20043 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From Facts to Folklore: Evaluating Large Language Models on Bengali Cultural Knowledge Chowdhury, Nafis Haque, Moinul Ahmed, Anika Tasnim, Nazia Shihab, Md. Istiak Hossain Rahman, Sajjadur Sadeque, Farig Computation and Language Machine Learning I.2.7 Recent progress in NLP research has demonstrated remarkable capabilities of large language models (LLMs) across a wide range of tasks. While recent multilingual benchmarks have advanced cultural evaluation for LLMs, critical gaps remain in capturing the nuances of low-resource cultures. Our work addresses these limitations through a Bengali Language Cultural Knowledge (BLanCK) dataset including folk traditions, culinary arts, and regional dialects. Our investigation of several multilingual language models shows that while these models perform well in non-cultural categories, they struggle significantly with cultural knowledge and performance improves substantially across all models when context is provided, emphasizing context-aware architectures and culturally curated training data. |
| title | From Facts to Folklore: Evaluating Large Language Models on Bengali Cultural Knowledge |
| topic | Computation and Language Machine Learning I.2.7 |
| url | https://arxiv.org/abs/2510.20043 |