Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2510.24081 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916067135193088 |
|---|---|
| author | Chang, Tyler A. Arnett, Catherine Sadallah, Abdelrahman Eldesokey, Abdelrahman Kashar, Abeer Daud, Abolade Olanihun, Abosede Grace Mohammed, Adamu Labaran Praise, Adeyemi Sharma, Adhikarimayum Meerajita Gupta, Aditi Merin, Adril Putra Bremang, Adwoa Iyigun, Afitab Simplício, Afonso Essouaied, Ahmed Chorana, Aicha Eppa, Akhil Oladipo, Akintunde Kuri, Akriti Ramesh, Akshay Dorkin, Aleksei Kondoro, Alfred Malengo Aji, Alham Fikri Çetintaş, Ali Eren Hanbury, Allan Dembele, Alou Niksarli, Alp Arroyo, Álvaro Bajand, Amin Khanna, Amol Chkhaidze, Ana Condez, Ana Carolina Hartl, Anamaria-Roberta Mkhonto, Andiswa Hoblitzell, Andrew Tran, Andrew Poulis, Angelos Majumder, Anirban Chaudhary, Anjali Vacalopoulou, Anna Wong, Annette Kuuipolani Kanahele Simonsen, Annika Kovalev, Anton Nayak, Anupam S, Ashvanth Lana, Ayodeji Purwarianti, Ayu Alhafni, Bashar Busole, Benedict Ghanem, Bernard Nathani, Bharti Đurić, Biljana Stojanovska Ogundipe, Blessing Agbonile, Bolaotan Bergsson, Bragi Fischer, Bruce Torres Tutar, Burak Çınar, Burcu Kane, Cade Udomcharoenchaikit, Can Helwe, Chadi Nerella, Chaithra Reddy Liu, Chen Cecilia Nwokolo, Chiamaka Homan, Christopher Sampebgo, Clément España-Bonet, Cristina Amol, Cynthia Lee, Daeyoep Smart, Dan Saattrup Arad, Dana Dzenhaliou, Daniil Choi, Dasol Liu, David Semedo, David Anugraha, David Popoola, Deborah Mataciunas, Deividas Nyaboke, Delphine Owusu, Dennis Kumar, Dhyuthy Krishna Tavares, Diogo Glória-Silva, Diogo Goyal, Divyanshu Lee, DongGeon Buchanan, E. Kelly Anajemba, Ebele Nwamaka Grace, Egonu Ngozi Mickel, Elena Herranen, Elias Acharya, Eliza Nisar, Eman Anand, Emile Habumuremyi, Emmanuel Ajiboye, Emuobonuvie Maria Yulianrifat, Eryawan Presma Adenuga, Esther Rudnicka, Ewa Itiola, Faith Butt, Faran Taimoor Sheikh, Fareeha Fayyaz Thekkekara, Fathima Haouari, Fatima Nsengiyumva, Faustin Ilasariya, Fenal Ashokbhai Tjiaranata, Filbert Aurelian Laakom, Firas Grasso, Francesca Periti, Francesco Orabona, Francesco Solomon, Gbenga Kayode Winata, Genta Indra Ngo, Gia Nghia Udhedhe-oze, Gloria Vinagre, Gonçalo Challagolla, Gopi Naga Sai Ram Urbizu-Garmendia, Gorka Vadithya, Gouthami Son, Guijin Abdykadyrova, Gulnaz Mohapatra, Gyan Swaroop Ullah, Hafeez Einarsson, Hafsteinn Hu, Hai Saffari, Hamidreza Zaidi, Hamza Zhang, Haopeng Shairah, Harethah Abu Vuong, Harry Kuulmets, Hele-Andra Patel, Hitesh Laxmichand Bouamor, Houda Yu, Hwanjo Debess, Iben Nyholm Deveci, İbrahim Ethem Hanif, Ikhlasul Akmal Cho, Ikhyun Vieira, Inês Calvo, Inês Manzi, Isaac Salifou, Ismael Illa Daud, Ismail Yusuf, Ismail Itzhak, Itay Zhelyazkov, Ivan Belashkin, Ivan Spada, Ivan Brinton, Jacob Isbarov, Jafar Čibej, Jaka Kocoń, Jan Cuhel, Jan Krito, Jauza Purbey, Jebish Za, Jennifer Mickel, Jennifer Kunz, Jenny Ratovondranto, Jessica Varsha, Jeyarajalingam Jeong, Jihae Dávalos, Jimena Tena Lee, Jinu Magalhães, João Yi, John Seon Keun Kim, Jongin Chataignon, Joseph Imperial, Joseph Marvin Thevakumar, Jubeerathan Land, Judith Alekseenko, Julia Jiang, Junchen Kim, Jungwhan Sirts, Kairit R, Kamesh V, Kamesh Tshinu, Kanda Kukk, Kätriin Ponkshe, Kaustubh Huseynova, Kavsar He, Ke Enevoldsen, Kenneth Alvarez, Kent Joshua Zaman, Kerem Mrini, Khalil Kyars, Kian Gour, Komal Lainitha, Krishnakumar Kruusmaa, Krister Mukherjee, Kunal Chouhan, Kusum Castro, Laura Porrino-Moscoso, Laura M. Nzambi, Lenny Sivi Za Choshen, Leshem Sencan, Levent Øvrelid, Lilja Alazraki, Lisa Jones, Loretta Oma Ehimen-Ugbede, Lovina Thevakumar, Luheerathan Thavarasa, Luxshan Malik, Mahnoor Keita, Mamadou K. Jangid, Mansi De Santis, Marco Garcia, Marcos Šuppa, Marek D'Ciofalo, Mariam Ojastu, Marii Attaullah, Marium Sikander, Maryam Narayan, Mausami Skandalis, Maximos Mehak, Mehak Bozkurt, Mehmet İlteriş Bayu, Melaku Velayuthan, Menan Vizo, Mhasilenuo Leventhal, Michael Marcińczuk, Michał Almasi, Mina Potočnjak, Mirna Bangera, Mithil Shafiei, Mohammadamin Ansari, Mohiba Sharma, Mridul Indoria, Mrityunjaya Rehman, Mughees Ur Habibi, Muhammad Ravi Shulthan Kolić, Murat Kınay, Murat Barkın Galant, Nada Rathore, Naina Singh Permpredanun, Naphat Maugin, Narada Norman, Nathalie Corrêa, Nicholas Kluge Ljubešić, Nikola Thomas, Nirmal de Silva, Nisansa Joshi, Nisheeth Ponkshe, Nitish Habash, Nizar Udeze, Nneoma Thomas, Noel Ligeti-Nagy, Noémi Coulibaly, Nouhoum Ogundepo, Odunayo Buliaminu, Odunayo Kareemat Fejiro, Oghojafor Godswill God'spraise, Okechukwu Samuel, Olanrewaju Oluwaseun, Olaoye Deborah Akindejoye, Olasoji Snissarenko, Olga Chiemezie, Onyinye Anulika Kınay, Orkun Tursun, Osman Joshua, Oyelade Oluwafemi Fiyinfoluwa, Oyesanmi Rodríguez, Pablo Gamallo, Pablo Arora, Palak Valente, Pedro Rupnik, Peter Ekiugbo, Philip Oghenesuowho Agarwal, Prakhar Sahoo, Pramit Prokopidis, Prokopis Niau-Puhipau, Pua Yahya, Quadri Mignone, Rachele Singhal, Raghav Raja, Rahul Kadiyala, Ram Mohan Rao Merx, Raphael Larsen, Rasmus Rajalakshmi, Ratnavel Ghosh, Rishav Oji, Romina Solis, Ron Kekeha Guerra, Rui Zawar, Rushikesh Bashir, Sa'ad Nasir Alzaabi, Saeed Sandeep, Sahil Batchu, Sai Pavan Kantareddy, Sai Sandeep Muzammil, Saleha Pranida, Salsabila Zahirah Buchanan, Sam Rutunda, Samuel Land, Sander Sulollari, Sarah Ali, Sardar Sapkota, Saroj Kengatharaiyer, Sarveswaran Tautvaisas, Saulius Sen, Sayambhu Banerjee, Sayantani Diarra, Sebastien Afolayan, Segun M, Senthilnathan Lee, Sewoong Shah, Shaan Venkitachalam, Shankar Djurabaeva, Sharifa Ibejih, Sharon Dutta, Shivanya Shomir Gupta, Siddhant Suárez, Silvia Paniagua Ahmadi, Sina Sukumar, Sivasuthan Song, Siyuan A, Snegha Sofianopoulos, Sokratis Simon, Sona Elza Benčina, Sonja Gvasalia, Sophie More, Sphurti Dragazis, Spyros Milosavljević, Stefan Kaufhold, Stephan P. S, Suba Alrashed, Sultan Ranathunga, Surangika Someya, Taiga Pungeršek, Taja Kuzman Haklay, Tal Jibril, Tasi'u Aoyama, Tatsuya Abashidze, Tea Cruz, Terenz Jomar Dela Blevins, Terra Nikas, Themistoklis Idoko, Theresa Do, Thu Mai Chubakov, Tilek Munda, Tina Owoeye, Tobiloba Gargiani, Tommaso Rathore, Uma Johannesen, Uni Ugwu, Uwuma Putra, Vallerie Alexandra Kumar, Vanya Bannihatti Arzt, Varvara Konovalov, Vasily Nedumpozhimana, Vasudevan Ondrejova, Viktoria Horbik, Viktoryia Kummitha, Vishnu Vardhan Reddy Dinić, Vuk Sewunetie, Walelign Wu, Winston Zhao, Xiaojing Diarra, Yacouba Nikankin, Yaniv Mathur, Yash Bagla, Yash Bangera, Yeshil Chen, Yixi Li, Yiyuan Xavier, Yolanda Belinkov, Yonatan Alyafeai, Zaid Batozargalova, Zhargal Shan, Zhengyang Tam, Zhi Rui Tang, Zilu Nadova, Zuzana Abbasi, Baber Biderman, Stella Stap, David Ataman, Duygu Schmidt, Fabian Gonen, Hila Wang, Jiayi Adelani, David Ifeoluwa |
| author_facet | Chang, Tyler A. Arnett, Catherine Sadallah, Abdelrahman Eldesokey, Abdelrahman Kashar, Abeer Daud, Abolade Olanihun, Abosede Grace Mohammed, Adamu Labaran Praise, Adeyemi Sharma, Adhikarimayum Meerajita Gupta, Aditi Merin, Adril Putra Bremang, Adwoa Iyigun, Afitab Simplício, Afonso Essouaied, Ahmed Chorana, Aicha Eppa, Akhil Oladipo, Akintunde Kuri, Akriti Ramesh, Akshay Dorkin, Aleksei Kondoro, Alfred Malengo Aji, Alham Fikri Çetintaş, Ali Eren Hanbury, Allan Dembele, Alou Niksarli, Alp Arroyo, Álvaro Bajand, Amin Khanna, Amol Chkhaidze, Ana Condez, Ana Carolina Hartl, Anamaria-Roberta Mkhonto, Andiswa Hoblitzell, Andrew Tran, Andrew Poulis, Angelos Majumder, Anirban Chaudhary, Anjali Vacalopoulou, Anna Wong, Annette Kuuipolani Kanahele Simonsen, Annika Kovalev, Anton Nayak, Anupam S, Ashvanth Lana, Ayodeji Purwarianti, Ayu Alhafni, Bashar Busole, Benedict Ghanem, Bernard Nathani, Bharti Đurić, Biljana Stojanovska Ogundipe, Blessing Agbonile, Bolaotan Bergsson, Bragi Fischer, Bruce Torres Tutar, Burak Çınar, Burcu Kane, Cade Udomcharoenchaikit, Can Helwe, Chadi Nerella, Chaithra Reddy Liu, Chen Cecilia Nwokolo, Chiamaka Homan, Christopher Sampebgo, Clément España-Bonet, Cristina Amol, Cynthia Lee, Daeyoep Smart, Dan Saattrup Arad, Dana Dzenhaliou, Daniil Choi, Dasol Liu, David Semedo, David Anugraha, David Popoola, Deborah Mataciunas, Deividas Nyaboke, Delphine Owusu, Dennis Kumar, Dhyuthy Krishna Tavares, Diogo Glória-Silva, Diogo Goyal, Divyanshu Lee, DongGeon Buchanan, E. Kelly Anajemba, Ebele Nwamaka Grace, Egonu Ngozi Mickel, Elena Herranen, Elias Acharya, Eliza Nisar, Eman Anand, Emile Habumuremyi, Emmanuel Ajiboye, Emuobonuvie Maria Yulianrifat, Eryawan Presma Adenuga, Esther Rudnicka, Ewa Itiola, Faith Butt, Faran Taimoor Sheikh, Fareeha Fayyaz Thekkekara, Fathima Haouari, Fatima Nsengiyumva, Faustin Ilasariya, Fenal Ashokbhai Tjiaranata, Filbert Aurelian Laakom, Firas Grasso, Francesca Periti, Francesco Orabona, Francesco Solomon, Gbenga Kayode Winata, Genta Indra Ngo, Gia Nghia Udhedhe-oze, Gloria Vinagre, Gonçalo Challagolla, Gopi Naga Sai Ram Urbizu-Garmendia, Gorka Vadithya, Gouthami Son, Guijin Abdykadyrova, Gulnaz Mohapatra, Gyan Swaroop Ullah, Hafeez Einarsson, Hafsteinn Hu, Hai Saffari, Hamidreza Zaidi, Hamza Zhang, Haopeng Shairah, Harethah Abu Vuong, Harry Kuulmets, Hele-Andra Patel, Hitesh Laxmichand Bouamor, Houda Yu, Hwanjo Debess, Iben Nyholm Deveci, İbrahim Ethem Hanif, Ikhlasul Akmal Cho, Ikhyun Vieira, Inês Calvo, Inês Manzi, Isaac Salifou, Ismael Illa Daud, Ismail Yusuf, Ismail Itzhak, Itay Zhelyazkov, Ivan Belashkin, Ivan Spada, Ivan Brinton, Jacob Isbarov, Jafar Čibej, Jaka Kocoń, Jan Cuhel, Jan Krito, Jauza Purbey, Jebish Za, Jennifer Mickel, Jennifer Kunz, Jenny Ratovondranto, Jessica Varsha, Jeyarajalingam Jeong, Jihae Dávalos, Jimena Tena Lee, Jinu Magalhães, João Yi, John Seon Keun Kim, Jongin Chataignon, Joseph Imperial, Joseph Marvin Thevakumar, Jubeerathan Land, Judith Alekseenko, Julia Jiang, Junchen Kim, Jungwhan Sirts, Kairit R, Kamesh V, Kamesh Tshinu, Kanda Kukk, Kätriin Ponkshe, Kaustubh Huseynova, Kavsar He, Ke Enevoldsen, Kenneth Alvarez, Kent Joshua Zaman, Kerem Mrini, Khalil Kyars, Kian Gour, Komal Lainitha, Krishnakumar Kruusmaa, Krister Mukherjee, Kunal Chouhan, Kusum Castro, Laura Porrino-Moscoso, Laura M. Nzambi, Lenny Sivi Za Choshen, Leshem Sencan, Levent Øvrelid, Lilja Alazraki, Lisa Jones, Loretta Oma Ehimen-Ugbede, Lovina Thevakumar, Luheerathan Thavarasa, Luxshan Malik, Mahnoor Keita, Mamadou K. Jangid, Mansi De Santis, Marco Garcia, Marcos Šuppa, Marek D'Ciofalo, Mariam Ojastu, Marii Attaullah, Marium Sikander, Maryam Narayan, Mausami Skandalis, Maximos Mehak, Mehak Bozkurt, Mehmet İlteriş Bayu, Melaku Velayuthan, Menan Vizo, Mhasilenuo Leventhal, Michael Marcińczuk, Michał Almasi, Mina Potočnjak, Mirna Bangera, Mithil Shafiei, Mohammadamin Ansari, Mohiba Sharma, Mridul Indoria, Mrityunjaya Rehman, Mughees Ur Habibi, Muhammad Ravi Shulthan Kolić, Murat Kınay, Murat Barkın Galant, Nada Rathore, Naina Singh Permpredanun, Naphat Maugin, Narada Norman, Nathalie Corrêa, Nicholas Kluge Ljubešić, Nikola Thomas, Nirmal de Silva, Nisansa Joshi, Nisheeth Ponkshe, Nitish Habash, Nizar Udeze, Nneoma Thomas, Noel Ligeti-Nagy, Noémi Coulibaly, Nouhoum Ogundepo, Odunayo Buliaminu, Odunayo Kareemat Fejiro, Oghojafor Godswill God'spraise, Okechukwu Samuel, Olanrewaju Oluwaseun, Olaoye Deborah Akindejoye, Olasoji Snissarenko, Olga Chiemezie, Onyinye Anulika Kınay, Orkun Tursun, Osman Joshua, Oyelade Oluwafemi Fiyinfoluwa, Oyesanmi Rodríguez, Pablo Gamallo, Pablo Arora, Palak Valente, Pedro Rupnik, Peter Ekiugbo, Philip Oghenesuowho Agarwal, Prakhar Sahoo, Pramit Prokopidis, Prokopis Niau-Puhipau, Pua Yahya, Quadri Mignone, Rachele Singhal, Raghav Raja, Rahul Kadiyala, Ram Mohan Rao Merx, Raphael Larsen, Rasmus Rajalakshmi, Ratnavel Ghosh, Rishav Oji, Romina Solis, Ron Kekeha Guerra, Rui Zawar, Rushikesh Bashir, Sa'ad Nasir Alzaabi, Saeed Sandeep, Sahil Batchu, Sai Pavan Kantareddy, Sai Sandeep Muzammil, Saleha Pranida, Salsabila Zahirah Buchanan, Sam Rutunda, Samuel Land, Sander Sulollari, Sarah Ali, Sardar Sapkota, Saroj Kengatharaiyer, Sarveswaran Tautvaisas, Saulius Sen, Sayambhu Banerjee, Sayantani Diarra, Sebastien Afolayan, Segun M, Senthilnathan Lee, Sewoong Shah, Shaan Venkitachalam, Shankar Djurabaeva, Sharifa Ibejih, Sharon Dutta, Shivanya Shomir Gupta, Siddhant Suárez, Silvia Paniagua Ahmadi, Sina Sukumar, Sivasuthan Song, Siyuan A, Snegha Sofianopoulos, Sokratis Simon, Sona Elza Benčina, Sonja Gvasalia, Sophie More, Sphurti Dragazis, Spyros Milosavljević, Stefan Kaufhold, Stephan P. S, Suba Alrashed, Sultan Ranathunga, Surangika Someya, Taiga Pungeršek, Taja Kuzman Haklay, Tal Jibril, Tasi'u Aoyama, Tatsuya Abashidze, Tea Cruz, Terenz Jomar Dela Blevins, Terra Nikas, Themistoklis Idoko, Theresa Do, Thu Mai Chubakov, Tilek Munda, Tina Owoeye, Tobiloba Gargiani, Tommaso Rathore, Uma Johannesen, Uni Ugwu, Uwuma Putra, Vallerie Alexandra Kumar, Vanya Bannihatti Arzt, Varvara Konovalov, Vasily Nedumpozhimana, Vasudevan Ondrejova, Viktoria Horbik, Viktoryia Kummitha, Vishnu Vardhan Reddy Dinić, Vuk Sewunetie, Walelign Wu, Winston Zhao, Xiaojing Diarra, Yacouba Nikankin, Yaniv Mathur, Yash Bagla, Yash Bangera, Yeshil Chen, Yixi Li, Yiyuan Xavier, Yolanda Belinkov, Yonatan Alyafeai, Zaid Batozargalova, Zhargal Shan, Zhengyang Tam, Zhi Rui Tang, Zilu Nadova, Zuzana Abbasi, Baber Biderman, Stella Stap, David Ataman, Duygu Schmidt, Fabian Gonen, Hila Wang, Jiayi Adelani, David Ifeoluwa |
| contents | To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we present Global PIQA, a participatory commonsense reasoning benchmark for over 100 languages, constructed by hand by over 350 researchers from over 65 countries around the world. The 141 language varieties in Global PIQA cover five continents, 19 language families, and 24 writing systems. In the non-parallel split of Global PIQA, over 50% of examples reference local foods, customs, traditions, or other culturally-specific elements. In the parallel split, we translate more "culturally agnostic" commonsense reasoning questions into 131 language varieties, for direct cross-lingual comparisons. In both splits, all examples have been verified by native speakers of the languages. We find that state-of-the-art LLMs perform well on Global PIQA in aggregate, but they exhibit weaker performance in lower-resource languages (e.g. up to a 68% accuracy gap between languages in the parallel split). Global PIQA highlights that in many languages and cultures, everyday knowledge remains an area for improvement in LLMs, alongside more widely-discussed capabilities such as complex reasoning and expert knowledge. Beyond its uses for LLM evaluation, Global PIQA provides a glimpse into the wide diversity of cultures in which human language is embedded. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_24081 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures Chang, Tyler A. Arnett, Catherine Sadallah, Abdelrahman Eldesokey, Abdelrahman Kashar, Abeer Daud, Abolade Olanihun, Abosede Grace Mohammed, Adamu Labaran Praise, Adeyemi Sharma, Adhikarimayum Meerajita Gupta, Aditi Merin, Adril Putra Bremang, Adwoa Iyigun, Afitab Simplício, Afonso Essouaied, Ahmed Chorana, Aicha Eppa, Akhil Oladipo, Akintunde Kuri, Akriti Ramesh, Akshay Dorkin, Aleksei Kondoro, Alfred Malengo Aji, Alham Fikri Çetintaş, Ali Eren Hanbury, Allan Dembele, Alou Niksarli, Alp Arroyo, Álvaro Bajand, Amin Khanna, Amol Chkhaidze, Ana Condez, Ana Carolina Hartl, Anamaria-Roberta Mkhonto, Andiswa Hoblitzell, Andrew Tran, Andrew Poulis, Angelos Majumder, Anirban Chaudhary, Anjali Vacalopoulou, Anna Wong, Annette Kuuipolani Kanahele Simonsen, Annika Kovalev, Anton Nayak, Anupam S, Ashvanth Lana, Ayodeji Purwarianti, Ayu Alhafni, Bashar Busole, Benedict Ghanem, Bernard Nathani, Bharti Đurić, Biljana Stojanovska Ogundipe, Blessing Agbonile, Bolaotan Bergsson, Bragi Fischer, Bruce Torres Tutar, Burak Çınar, Burcu Kane, Cade Udomcharoenchaikit, Can Helwe, Chadi Nerella, Chaithra Reddy Liu, Chen Cecilia Nwokolo, Chiamaka Homan, Christopher Sampebgo, Clément España-Bonet, Cristina Amol, Cynthia Lee, Daeyoep Smart, Dan Saattrup Arad, Dana Dzenhaliou, Daniil Choi, Dasol Liu, David Semedo, David Anugraha, David Popoola, Deborah Mataciunas, Deividas Nyaboke, Delphine Owusu, Dennis Kumar, Dhyuthy Krishna Tavares, Diogo Glória-Silva, Diogo Goyal, Divyanshu Lee, DongGeon Buchanan, E. Kelly Anajemba, Ebele Nwamaka Grace, Egonu Ngozi Mickel, Elena Herranen, Elias Acharya, Eliza Nisar, Eman Anand, Emile Habumuremyi, Emmanuel Ajiboye, Emuobonuvie Maria Yulianrifat, Eryawan Presma Adenuga, Esther Rudnicka, Ewa Itiola, Faith Butt, Faran Taimoor Sheikh, Fareeha Fayyaz Thekkekara, Fathima Haouari, Fatima Nsengiyumva, Faustin Ilasariya, Fenal Ashokbhai Tjiaranata, Filbert Aurelian Laakom, Firas Grasso, Francesca Periti, Francesco Orabona, Francesco Solomon, Gbenga Kayode Winata, Genta Indra Ngo, Gia Nghia Udhedhe-oze, Gloria Vinagre, Gonçalo Challagolla, Gopi Naga Sai Ram Urbizu-Garmendia, Gorka Vadithya, Gouthami Son, Guijin Abdykadyrova, Gulnaz Mohapatra, Gyan Swaroop Ullah, Hafeez Einarsson, Hafsteinn Hu, Hai Saffari, Hamidreza Zaidi, Hamza Zhang, Haopeng Shairah, Harethah Abu Vuong, Harry Kuulmets, Hele-Andra Patel, Hitesh Laxmichand Bouamor, Houda Yu, Hwanjo Debess, Iben Nyholm Deveci, İbrahim Ethem Hanif, Ikhlasul Akmal Cho, Ikhyun Vieira, Inês Calvo, Inês Manzi, Isaac Salifou, Ismael Illa Daud, Ismail Yusuf, Ismail Itzhak, Itay Zhelyazkov, Ivan Belashkin, Ivan Spada, Ivan Brinton, Jacob Isbarov, Jafar Čibej, Jaka Kocoń, Jan Cuhel, Jan Krito, Jauza Purbey, Jebish Za, Jennifer Mickel, Jennifer Kunz, Jenny Ratovondranto, Jessica Varsha, Jeyarajalingam Jeong, Jihae Dávalos, Jimena Tena Lee, Jinu Magalhães, João Yi, John Seon Keun Kim, Jongin Chataignon, Joseph Imperial, Joseph Marvin Thevakumar, Jubeerathan Land, Judith Alekseenko, Julia Jiang, Junchen Kim, Jungwhan Sirts, Kairit R, Kamesh V, Kamesh Tshinu, Kanda Kukk, Kätriin Ponkshe, Kaustubh Huseynova, Kavsar He, Ke Enevoldsen, Kenneth Alvarez, Kent Joshua Zaman, Kerem Mrini, Khalil Kyars, Kian Gour, Komal Lainitha, Krishnakumar Kruusmaa, Krister Mukherjee, Kunal Chouhan, Kusum Castro, Laura Porrino-Moscoso, Laura M. Nzambi, Lenny Sivi Za Choshen, Leshem Sencan, Levent Øvrelid, Lilja Alazraki, Lisa Jones, Loretta Oma Ehimen-Ugbede, Lovina Thevakumar, Luheerathan Thavarasa, Luxshan Malik, Mahnoor Keita, Mamadou K. Jangid, Mansi De Santis, Marco Garcia, Marcos Šuppa, Marek D'Ciofalo, Mariam Ojastu, Marii Attaullah, Marium Sikander, Maryam Narayan, Mausami Skandalis, Maximos Mehak, Mehak Bozkurt, Mehmet İlteriş Bayu, Melaku Velayuthan, Menan Vizo, Mhasilenuo Leventhal, Michael Marcińczuk, Michał Almasi, Mina Potočnjak, Mirna Bangera, Mithil Shafiei, Mohammadamin Ansari, Mohiba Sharma, Mridul Indoria, Mrityunjaya Rehman, Mughees Ur Habibi, Muhammad Ravi Shulthan Kolić, Murat Kınay, Murat Barkın Galant, Nada Rathore, Naina Singh Permpredanun, Naphat Maugin, Narada Norman, Nathalie Corrêa, Nicholas Kluge Ljubešić, Nikola Thomas, Nirmal de Silva, Nisansa Joshi, Nisheeth Ponkshe, Nitish Habash, Nizar Udeze, Nneoma Thomas, Noel Ligeti-Nagy, Noémi Coulibaly, Nouhoum Ogundepo, Odunayo Buliaminu, Odunayo Kareemat Fejiro, Oghojafor Godswill God'spraise, Okechukwu Samuel, Olanrewaju Oluwaseun, Olaoye Deborah Akindejoye, Olasoji Snissarenko, Olga Chiemezie, Onyinye Anulika Kınay, Orkun Tursun, Osman Joshua, Oyelade Oluwafemi Fiyinfoluwa, Oyesanmi Rodríguez, Pablo Gamallo, Pablo Arora, Palak Valente, Pedro Rupnik, Peter Ekiugbo, Philip Oghenesuowho Agarwal, Prakhar Sahoo, Pramit Prokopidis, Prokopis Niau-Puhipau, Pua Yahya, Quadri Mignone, Rachele Singhal, Raghav Raja, Rahul Kadiyala, Ram Mohan Rao Merx, Raphael Larsen, Rasmus Rajalakshmi, Ratnavel Ghosh, Rishav Oji, Romina Solis, Ron Kekeha Guerra, Rui Zawar, Rushikesh Bashir, Sa'ad Nasir Alzaabi, Saeed Sandeep, Sahil Batchu, Sai Pavan Kantareddy, Sai Sandeep Muzammil, Saleha Pranida, Salsabila Zahirah Buchanan, Sam Rutunda, Samuel Land, Sander Sulollari, Sarah Ali, Sardar Sapkota, Saroj Kengatharaiyer, Sarveswaran Tautvaisas, Saulius Sen, Sayambhu Banerjee, Sayantani Diarra, Sebastien Afolayan, Segun M, Senthilnathan Lee, Sewoong Shah, Shaan Venkitachalam, Shankar Djurabaeva, Sharifa Ibejih, Sharon Dutta, Shivanya Shomir Gupta, Siddhant Suárez, Silvia Paniagua Ahmadi, Sina Sukumar, Sivasuthan Song, Siyuan A, Snegha Sofianopoulos, Sokratis Simon, Sona Elza Benčina, Sonja Gvasalia, Sophie More, Sphurti Dragazis, Spyros Milosavljević, Stefan Kaufhold, Stephan P. S, Suba Alrashed, Sultan Ranathunga, Surangika Someya, Taiga Pungeršek, Taja Kuzman Haklay, Tal Jibril, Tasi'u Aoyama, Tatsuya Abashidze, Tea Cruz, Terenz Jomar Dela Blevins, Terra Nikas, Themistoklis Idoko, Theresa Do, Thu Mai Chubakov, Tilek Munda, Tina Owoeye, Tobiloba Gargiani, Tommaso Rathore, Uma Johannesen, Uni Ugwu, Uwuma Putra, Vallerie Alexandra Kumar, Vanya Bannihatti Arzt, Varvara Konovalov, Vasily Nedumpozhimana, Vasudevan Ondrejova, Viktoria Horbik, Viktoryia Kummitha, Vishnu Vardhan Reddy Dinić, Vuk Sewunetie, Walelign Wu, Winston Zhao, Xiaojing Diarra, Yacouba Nikankin, Yaniv Mathur, Yash Bagla, Yash Bangera, Yeshil Chen, Yixi Li, Yiyuan Xavier, Yolanda Belinkov, Yonatan Alyafeai, Zaid Batozargalova, Zhargal Shan, Zhengyang Tam, Zhi Rui Tang, Zilu Nadova, Zuzana Abbasi, Baber Biderman, Stella Stap, David Ataman, Duygu Schmidt, Fabian Gonen, Hila Wang, Jiayi Adelani, David Ifeoluwa Computation and Language To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we present Global PIQA, a participatory commonsense reasoning benchmark for over 100 languages, constructed by hand by over 350 researchers from over 65 countries around the world. The 141 language varieties in Global PIQA cover five continents, 19 language families, and 24 writing systems. In the non-parallel split of Global PIQA, over 50% of examples reference local foods, customs, traditions, or other culturally-specific elements. In the parallel split, we translate more "culturally agnostic" commonsense reasoning questions into 131 language varieties, for direct cross-lingual comparisons. In both splits, all examples have been verified by native speakers of the languages. We find that state-of-the-art LLMs perform well on Global PIQA in aggregate, but they exhibit weaker performance in lower-resource languages (e.g. up to a 68% accuracy gap between languages in the parallel split). Global PIQA highlights that in many languages and cultures, everyday knowledge remains an area for improvement in LLMs, alongside more widely-discussed capabilities such as complex reasoning and expert knowledge. Beyond its uses for LLM evaluation, Global PIQA provides a glimpse into the wide diversity of cultures in which human language is embedded. |
| title | Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.24081 |