Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2511.20686 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912729502056448 |
|---|---|
| author | Lim, Chae-Gyun Han, Seung-Ho Byun, EunYoung Han, Jeongyun Cho, Soohyun Joo, Eojin Kim, Heehyeon Kim, Sieun Lee, Juhoon Lee, Hyunsoo Lee, Dongkun Hyeon, Jonghwan Hwang, Yechan Lee, Young-Jun Lee, Kyeongryul An, Minhyeong Ahn, Hyunjun Son, Jeongwoo Park, Junho Yoon, Donggyu Kim, Taehyung Kim, Jeemin Choi, Dasom Lee, Kwangyoung Lim, Hyunseung Jung, Yeohyun Hong, Jongok Nam, Sooyohn Park, Joonyoung Na, Sungmin Choi, Yubin Choi, Jeanne Hong, Yoojin Jang, Sueun Seo, Youngseok Park, Somin Jo, Seoungung Chae, Wonhye Jo, Yeeun Kim, Eunyoung Whang, Joyce Jiyoung Hong, HwaJung Seering, Joseph Lee, Uichin Kim, Juho Choi, Sunna Ko, Seokyeon Kim, Taeho Kim, Kyunghoon Ha, Myungsik Lee, So Jung Hwang, Jemin Kwak, JoonHo Choi, Ho-Jin |
| author_facet | Lim, Chae-Gyun Han, Seung-Ho Byun, EunYoung Han, Jeongyun Cho, Soohyun Joo, Eojin Kim, Heehyeon Kim, Sieun Lee, Juhoon Lee, Hyunsoo Lee, Dongkun Hyeon, Jonghwan Hwang, Yechan Lee, Young-Jun Lee, Kyeongryul An, Minhyeong Ahn, Hyunjun Son, Jeongwoo Park, Junho Yoon, Donggyu Kim, Taehyung Kim, Jeemin Choi, Dasom Lee, Kwangyoung Lim, Hyunseung Jung, Yeohyun Hong, Jongok Nam, Sooyohn Park, Joonyoung Na, Sungmin Choi, Yubin Choi, Jeanne Hong, Yoojin Jang, Sueun Seo, Youngseok Park, Somin Jo, Seoungung Chae, Wonhye Jo, Yeeun Kim, Eunyoung Whang, Joyce Jiyoung Hong, HwaJung Seering, Joseph Lee, Uichin Kim, Juho Choi, Sunna Ko, Seokyeon Kim, Taeho Kim, Kyunghoon Ha, Myungsik Lee, So Jung Hwang, Jemin Kwak, JoonHo Choi, Ho-Jin |
| contents | The rapid evolution of generative AI necessitates robust safety evaluations. However, current safety datasets are predominantly English-centric, failing to capture specific risks in non-English, socio-cultural contexts such as Korean, and are often limited to the text modality. To address this gap, we introduce AssurAI, a new quality-controlled Korean multimodal dataset for evaluating the safety of generative AI. First, we define a taxonomy of 35 distinct AI risk factors, adapted from established frameworks by a multidisciplinary expert group to cover both universal harms and relevance to the Korean socio-cultural context. Second, leveraging this taxonomy, we construct and release AssurAI, a large-scale Korean multimodal dataset comprising 11,480 instances across text, image, video, and audio. Third, we apply the rigorous quality control process used to ensure data integrity, featuring a two-phase construction (i.e., expert-led seeding and crowdsourced scaling), triple independent annotation, and an iterative expert red-teaming loop. Our pilot study validates AssurAI's effectiveness in assessing the safety of recent LLMs. We release AssurAI to the public to facilitate the development of safer and more reliable generative AI systems for the Korean community. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_20686 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AssurAI: Experience with Constructing Korean Socio-cultural Datasets to Discover Potential Risks of Generative AI Lim, Chae-Gyun Han, Seung-Ho Byun, EunYoung Han, Jeongyun Cho, Soohyun Joo, Eojin Kim, Heehyeon Kim, Sieun Lee, Juhoon Lee, Hyunsoo Lee, Dongkun Hyeon, Jonghwan Hwang, Yechan Lee, Young-Jun Lee, Kyeongryul An, Minhyeong Ahn, Hyunjun Son, Jeongwoo Park, Junho Yoon, Donggyu Kim, Taehyung Kim, Jeemin Choi, Dasom Lee, Kwangyoung Lim, Hyunseung Jung, Yeohyun Hong, Jongok Nam, Sooyohn Park, Joonyoung Na, Sungmin Choi, Yubin Choi, Jeanne Hong, Yoojin Jang, Sueun Seo, Youngseok Park, Somin Jo, Seoungung Chae, Wonhye Jo, Yeeun Kim, Eunyoung Whang, Joyce Jiyoung Hong, HwaJung Seering, Joseph Lee, Uichin Kim, Juho Choi, Sunna Ko, Seokyeon Kim, Taeho Kim, Kyunghoon Ha, Myungsik Lee, So Jung Hwang, Jemin Kwak, JoonHo Choi, Ho-Jin Artificial Intelligence Computers and Society Machine Learning The rapid evolution of generative AI necessitates robust safety evaluations. However, current safety datasets are predominantly English-centric, failing to capture specific risks in non-English, socio-cultural contexts such as Korean, and are often limited to the text modality. To address this gap, we introduce AssurAI, a new quality-controlled Korean multimodal dataset for evaluating the safety of generative AI. First, we define a taxonomy of 35 distinct AI risk factors, adapted from established frameworks by a multidisciplinary expert group to cover both universal harms and relevance to the Korean socio-cultural context. Second, leveraging this taxonomy, we construct and release AssurAI, a large-scale Korean multimodal dataset comprising 11,480 instances across text, image, video, and audio. Third, we apply the rigorous quality control process used to ensure data integrity, featuring a two-phase construction (i.e., expert-led seeding and crowdsourced scaling), triple independent annotation, and an iterative expert red-teaming loop. Our pilot study validates AssurAI's effectiveness in assessing the safety of recent LLMs. We release AssurAI to the public to facilitate the development of safer and more reliable generative AI systems for the Korean community. |
| title | AssurAI: Experience with Constructing Korean Socio-cultural Datasets to Discover Potential Risks of Generative AI |
| topic | Artificial Intelligence Computers and Society Machine Learning |
| url | https://arxiv.org/abs/2511.20686 |