Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.12705 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918012720775168 |
|---|---|
| author | Winata, Genta Indra Hudi, Frederikus Irawan, Patrick Amadeus Anugraha, David Putri, Rifki Afina Wang, Yutong Nohejl, Adam Prathama, Ubaidillah Ariq Ousidhoum, Nedjma Amriani, Afifa Rzayev, Anar Das, Anirban Pramodya, Ashmari Adila, Aulia Wilie, Bryan Mawalim, Candy Olivia Cheng, Ching Lam Abolade, Daud Chersoni, Emmanuele Santus, Enrico Ikhwantri, Fariz Kuwanto, Garry Zhao, Hanyang Wibowo, Haryo Akbarianto Lovenia, Holy Cruz, Jan Christian Blaise Putra, Jan Wira Gotama Myung, Junho Susanto, Lucky Machin, Maria Angelica Riera Zhukova, Marina Anugraha, Michael Adilazuarda, Muhammad Farid Santosa, Natasha Limkonchotiwat, Peerat Dabre, Raj Audino, Rio Alexander Cahyawijaya, Samuel Zhang, Shi-Xiong Salim, Stephanie Yulia Zhou, Yi Gui, Yinxuan Adelani, David Ifeoluwa Lee, En-Shiun Annie Okada, Shogo Purwarianti, Ayu Aji, Alham Fikri Watanabe, Taro Wijaya, Derry Tanti Oh, Alice Ngo, Chong-Wah |
| author_facet | Winata, Genta Indra Hudi, Frederikus Irawan, Patrick Amadeus Anugraha, David Putri, Rifki Afina Wang, Yutong Nohejl, Adam Prathama, Ubaidillah Ariq Ousidhoum, Nedjma Amriani, Afifa Rzayev, Anar Das, Anirban Pramodya, Ashmari Adila, Aulia Wilie, Bryan Mawalim, Candy Olivia Cheng, Ching Lam Abolade, Daud Chersoni, Emmanuele Santus, Enrico Ikhwantri, Fariz Kuwanto, Garry Zhao, Hanyang Wibowo, Haryo Akbarianto Lovenia, Holy Cruz, Jan Christian Blaise Putra, Jan Wira Gotama Myung, Junho Susanto, Lucky Machin, Maria Angelica Riera Zhukova, Marina Anugraha, Michael Adilazuarda, Muhammad Farid Santosa, Natasha Limkonchotiwat, Peerat Dabre, Raj Audino, Rio Alexander Cahyawijaya, Samuel Zhang, Shi-Xiong Salim, Stephanie Yulia Zhou, Yi Gui, Yinxuan Adelani, David Ifeoluwa Lee, En-Shiun Annie Okada, Shogo Purwarianti, Ayu Aji, Alham Fikri Watanabe, Taro Wijaya, Derry Tanti Oh, Alice Ngo, Chong-Wah |
| contents | Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate their understanding of such knowledge, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. It includes tasks for identifying dish names and their origins. We provide evaluation datasets in two sizes (12k and 60k instances) alongside a training dataset (1 million instances). Our findings show that while VLMs perform better with correct location context, they struggle with adversarial contexts and predicting specific regional cuisines and languages. To support future research, we release a knowledge base with annotated food entries and images along with the VQA data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_12705 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines Winata, Genta Indra Hudi, Frederikus Irawan, Patrick Amadeus Anugraha, David Putri, Rifki Afina Wang, Yutong Nohejl, Adam Prathama, Ubaidillah Ariq Ousidhoum, Nedjma Amriani, Afifa Rzayev, Anar Das, Anirban Pramodya, Ashmari Adila, Aulia Wilie, Bryan Mawalim, Candy Olivia Cheng, Ching Lam Abolade, Daud Chersoni, Emmanuele Santus, Enrico Ikhwantri, Fariz Kuwanto, Garry Zhao, Hanyang Wibowo, Haryo Akbarianto Lovenia, Holy Cruz, Jan Christian Blaise Putra, Jan Wira Gotama Myung, Junho Susanto, Lucky Machin, Maria Angelica Riera Zhukova, Marina Anugraha, Michael Adilazuarda, Muhammad Farid Santosa, Natasha Limkonchotiwat, Peerat Dabre, Raj Audino, Rio Alexander Cahyawijaya, Samuel Zhang, Shi-Xiong Salim, Stephanie Yulia Zhou, Yi Gui, Yinxuan Adelani, David Ifeoluwa Lee, En-Shiun Annie Okada, Shogo Purwarianti, Ayu Aji, Alham Fikri Watanabe, Taro Wijaya, Derry Tanti Oh, Alice Ngo, Chong-Wah Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate their understanding of such knowledge, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. It includes tasks for identifying dish names and their origins. We provide evaluation datasets in two sizes (12k and 60k instances) alongside a training dataset (1 million instances). Our findings show that while VLMs perform better with correct location context, they struggle with adversarial contexts and predicting specific regional cuisines and languages. To support future research, we release a knowledge base with annotated food entries and images along with the VQA data. |
| title | WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines |
| topic | Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.12705 |