Getting Serious about Humor: Crafting Humor Datasets with Unfunny Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909228521750528 |
|---|---|
| author | Horvitz, Zachary Chen, Jingru Aditya, Rahul Srivastava, Harshvardhan West, Robert Yu, Zhou McKeown, Kathleen |
| author_facet | Horvitz, Zachary Chen, Jingru Aditya, Rahul Srivastava, Harshvardhan West, Robert Yu, Zhou McKeown, Kathleen |
| contents | Humor is a fundamental facet of human cognition and interaction. Yet, despite recent advances in natural language processing, humor detection remains a challenging task that is complicated by the scarcity of datasets that pair humorous texts with similar non-humorous counterparts. In our work, we investigate whether large language models (LLMs), can generate synthetic data for humor detection via editing texts. We benchmark LLMs on an existing human dataset and show that current LLMs display an impressive ability to 'unfun' jokes, as judged by humans and as measured on the downstream task of humor detection. We extend our approach to a code-mixed English-Hindi humor dataset, where we find that GPT-4's synthetic data is highly rated by bilingual annotators and provides challenging adversarial examples for humor classifiers. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_00794 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Getting Serious about Humor: Crafting Humor Datasets with Unfunny Large Language Models Horvitz, Zachary Chen, Jingru Aditya, Rahul Srivastava, Harshvardhan West, Robert Yu, Zhou McKeown, Kathleen Computation and Language Artificial Intelligence Machine Learning Humor is a fundamental facet of human cognition and interaction. Yet, despite recent advances in natural language processing, humor detection remains a challenging task that is complicated by the scarcity of datasets that pair humorous texts with similar non-humorous counterparts. In our work, we investigate whether large language models (LLMs), can generate synthetic data for humor detection via editing texts. We benchmark LLMs on an existing human dataset and show that current LLMs display an impressive ability to 'unfun' jokes, as judged by humans and as measured on the downstream task of humor detection. We extend our approach to a code-mixed English-Hindi humor dataset, where we find that GPT-4's synthetic data is highly rated by bilingual annotators and provides challenging adversarial examples for humor classifiers. |
| title | Getting Serious about Humor: Crafting Humor Datasets with Unfunny Large Language Models |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2403.00794 |