Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.00658 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912589977485312 |
|---|---|
| author | Xiong, Lang Gao, Raina Jeong, Alyssa Fu, Yicheng O'Brien, Sean Sharma, Vasu Zhu, Kevin |
| author_facet | Xiong, Lang Gao, Raina Jeong, Alyssa Fu, Yicheng O'Brien, Sean Sharma, Vasu Zhu, Kevin |
| contents | Sarcasm is a form of humor where expressions convey meanings opposite to their literal interpretations. Classifying and generating sarcasm using large language models is vital for interpreting human communication. Sarcasm poses challenges for computational models, due to its nuanced nature. We introduce Sarc7, a benchmark that classifies 7 types of sarcasm: self-deprecating, brooding, deadpan, polite, obnoxious, raging, and manic by annotating entries of the MUStARD dataset. Classification was evaluated using zero-shot, few-shot, chain-of-thought (CoT), and a novel emotion-based prompting technique. We propose an emotion-based generation method developed by identifying key components of sarcasm-incongruity, shock value, and context dependency. Our classification experiments show that Gemini 2.5, using emotion-based prompting, outperforms other setups with an F1 score of 0.3664. Human evaluators preferred our emotion-based prompting, with 38.46% more successful generations than zero-shot prompting. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_00658 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Sarc7: Evaluating Sarcasm Detection and Generation with Seven Types and Emotion-Informed Techniques Xiong, Lang Gao, Raina Jeong, Alyssa Fu, Yicheng O'Brien, Sean Sharma, Vasu Zhu, Kevin Computation and Language Artificial Intelligence Sarcasm is a form of humor where expressions convey meanings opposite to their literal interpretations. Classifying and generating sarcasm using large language models is vital for interpreting human communication. Sarcasm poses challenges for computational models, due to its nuanced nature. We introduce Sarc7, a benchmark that classifies 7 types of sarcasm: self-deprecating, brooding, deadpan, polite, obnoxious, raging, and manic by annotating entries of the MUStARD dataset. Classification was evaluated using zero-shot, few-shot, chain-of-thought (CoT), and a novel emotion-based prompting technique. We propose an emotion-based generation method developed by identifying key components of sarcasm-incongruity, shock value, and context dependency. Our classification experiments show that Gemini 2.5, using emotion-based prompting, outperforms other setups with an F1 score of 0.3664. Human evaluators preferred our emotion-based prompting, with 38.46% more successful generations than zero-shot prompting. |
| title | Sarc7: Evaluating Sarcasm Detection and Generation with Seven Types and Emotion-Informed Techniques |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2506.00658 |