Mathematics Isn't Culture-Free: Probing Cultural Gaps via Entity and Scenario Perturbations
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914125899104256 |
|---|---|
| author | Tomar, Aditya Sahoo, Nihar Ranjan Mittal, Ashish Murthy, Rudra Bhattacharyya, Pushpak |
| author_facet | Tomar, Aditya Sahoo, Nihar Ranjan Mittal, Ashish Murthy, Rudra Bhattacharyya, Pushpak |
| contents | Although mathematics is often considered culturally neutral, the way mathematical problems are presented can carry implicit cultural context. Existing benchmarks like GSM8K are predominantly rooted in Western norms, including names, currencies, and everyday scenarios. In this work, we create culturally adapted variants of the GSM8K test set for five regions Africa, India, China, Korea, and Japan using prompt-based transformations followed by manual verification. We evaluate six large language models (LLMs), ranging from 8B to 72B parameters, across five prompting strategies to assess their robustness to cultural variation in math problem presentation. Our findings reveal a consistent performance gap: models perform best on the original US-centric dataset and comparatively worse on culturally adapted versions. However, models with reasoning capabilities are more resilient to these shifts, suggesting that deeper reasoning helps bridge cultural presentation gaps in mathematical tasks |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_00883 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mathematics Isn't Culture-Free: Probing Cultural Gaps via Entity and Scenario Perturbations Tomar, Aditya Sahoo, Nihar Ranjan Mittal, Ashish Murthy, Rudra Bhattacharyya, Pushpak Computation and Language Although mathematics is often considered culturally neutral, the way mathematical problems are presented can carry implicit cultural context. Existing benchmarks like GSM8K are predominantly rooted in Western norms, including names, currencies, and everyday scenarios. In this work, we create culturally adapted variants of the GSM8K test set for five regions Africa, India, China, Korea, and Japan using prompt-based transformations followed by manual verification. We evaluate six large language models (LLMs), ranging from 8B to 72B parameters, across five prompting strategies to assess their robustness to cultural variation in math problem presentation. Our findings reveal a consistent performance gap: models perform best on the original US-centric dataset and comparatively worse on culturally adapted versions. However, models with reasoning capabilities are more resilient to these shifts, suggesting that deeper reasoning helps bridge cultural presentation gaps in mathematical tasks |
| title | Mathematics Isn't Culture-Free: Probing Cultural Gaps via Entity and Scenario Perturbations |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2507.00883 |