Let's Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models' Understanding of Sports
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915530434150400 |
|---|---|
| author | Singh, Punit Kumar Kumar, Nishant Ghosh, Akash Pasad, Kunal Soni, Khushi Jaishwal, Manisha Saha, Sriparna Alfarozi, Syukron Abu Ishaq Abagissa, Asres Temam Pasupa, Kitsuchart Yang, Haiqin Moreno, Jose G |
| author_facet | Singh, Punit Kumar Kumar, Nishant Ghosh, Akash Pasad, Kunal Soni, Khushi Jaishwal, Manisha Saha, Sriparna Alfarozi, Syukron Abu Ishaq Abagissa, Asres Temam Pasupa, Kitsuchart Yang, Haiqin Moreno, Jose G |
| contents | Language Models (LMs) are primarily evaluated on globally popular sports, often overlooking regional and indigenous sporting traditions. To address this gap, we introduce \textbf{\textit{CultSportQA}}, a benchmark designed to assess LMs' understanding of traditional sports across 60 countries and 6 continents, encompassing four distinct cultural categories. The dataset features 33,000 multiple-choice questions (MCQs) across text and image modalities, each of which is categorized into three key types: history-based, rule-based, and scenario-based. To evaluate model performance, we employ zero-shot, few-shot, and chain-of-thought (CoT) prompting across a diverse set of Large Language Models (LLMs), Small Language Models (SLMs), and Multimodal Large Language Models (MLMs). By providing a comprehensive multilingual and multicultural sports benchmark, \textbf{\textit{CultSportQA}} establishes a new standard for assessing AI's ability to understand and reason about traditional sports. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_01247 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Let's Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models' Understanding of Sports Singh, Punit Kumar Kumar, Nishant Ghosh, Akash Pasad, Kunal Soni, Khushi Jaishwal, Manisha Saha, Sriparna Alfarozi, Syukron Abu Ishaq Abagissa, Asres Temam Pasupa, Kitsuchart Yang, Haiqin Moreno, Jose G Computation and Language Artificial Intelligence Language Models (LMs) are primarily evaluated on globally popular sports, often overlooking regional and indigenous sporting traditions. To address this gap, we introduce \textbf{\textit{CultSportQA}}, a benchmark designed to assess LMs' understanding of traditional sports across 60 countries and 6 continents, encompassing four distinct cultural categories. The dataset features 33,000 multiple-choice questions (MCQs) across text and image modalities, each of which is categorized into three key types: history-based, rule-based, and scenario-based. To evaluate model performance, we employ zero-shot, few-shot, and chain-of-thought (CoT) prompting across a diverse set of Large Language Models (LLMs), Small Language Models (SLMs), and Multimodal Large Language Models (MLMs). By providing a comprehensive multilingual and multicultural sports benchmark, \textbf{\textit{CultSportQA}} establishes a new standard for assessing AI's ability to understand and reason about traditional sports. |
| title | Let's Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models' Understanding of Sports |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2510.01247 |