Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2406.12754 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917698409070592 |
|---|---|
| author | He, Ruiqi He, Yushu Bai, Longju Liu, Jiarui Sun, Zhenjie Tang, Zenghao Wang, He Xia, Hanchen Deng, Naihao |
| author_facet | He, Ruiqi He, Yushu Bai, Longju Liu, Jiarui Sun, Zhenjie Tang, Zenghao Wang, He Xia, Hanchen Deng, Naihao |
| contents | Existing humor datasets and evaluations predominantly focus on English, lacking resources for culturally nuanced humor in non-English languages like Chinese. To address this gap, we construct Chumor, a dataset sourced from Ruo Zhi Ba (RZB), a Chinese Reddit-like platform dedicated to sharing intellectually challenging and culturally specific jokes. We annotate explanations for each joke and evaluate human explanations against two state-of-the-art LLMs, GPT-4o and ERNIE Bot, through A/B testing by native Chinese speakers. Our evaluation shows that Chumor is challenging even for SOTA LLMs, and the human explanations for Chumor jokes are significantly better than explanations generated by the LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_12754 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Chumor 1.0: A Truly Funny and Challenging Chinese Humor Understanding Dataset from Ruo Zhi Ba He, Ruiqi He, Yushu Bai, Longju Liu, Jiarui Sun, Zhenjie Tang, Zenghao Wang, He Xia, Hanchen Deng, Naihao Computation and Language Artificial Intelligence Existing humor datasets and evaluations predominantly focus on English, lacking resources for culturally nuanced humor in non-English languages like Chinese. To address this gap, we construct Chumor, a dataset sourced from Ruo Zhi Ba (RZB), a Chinese Reddit-like platform dedicated to sharing intellectually challenging and culturally specific jokes. We annotate explanations for each joke and evaluate human explanations against two state-of-the-art LLMs, GPT-4o and ERNIE Bot, through A/B testing by native Chinese speakers. Our evaluation shows that Chumor is challenging even for SOTA LLMs, and the human explanations for Chumor jokes are significantly better than explanations generated by the LLMs. |
| title | Chumor 1.0: A Truly Funny and Challenging Chinese Humor Understanding Dataset from Ruo Zhi Ba |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2406.12754 |