Think Like a Person Before Responding: A Multi-Faceted Evaluation of Persona-Guided LLMs for Countering Hate
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908393424289792 |
|---|---|
| author | Ngueajio, Mikel K. Plaza-del-Arco, Flor Miriam Chung, Yi-Ling Rawat, Danda B. Curry, Amanda Cercas |
| author_facet | Ngueajio, Mikel K. Plaza-del-Arco, Flor Miriam Chung, Yi-Ling Rawat, Danda B. Curry, Amanda Cercas |
| contents | Automated counter-narratives (CN) offer a promising strategy for mitigating online hate speech, yet concerns about their affective tone, accessibility, and ethical risks remain. We propose a framework for evaluating Large Language Model (LLM)-generated CNs across four dimensions: persona framing, verbosity and readability, affective tone, and ethical robustness. Using GPT-4o-Mini, Cohere's CommandR-7B, and Meta's LLaMA 3.1-70B, we assess three prompting strategies on the MT-Conan and HatEval datasets. Our findings reveal that LLM-generated CNs are often verbose and adapted for people with college-level literacy, limiting their accessibility. While emotionally guided prompts yield more empathetic and readable responses, there remain concerns surrounding safety and effectiveness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_04043 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Think Like a Person Before Responding: A Multi-Faceted Evaluation of Persona-Guided LLMs for Countering Hate Ngueajio, Mikel K. Plaza-del-Arco, Flor Miriam Chung, Yi-Ling Rawat, Danda B. Curry, Amanda Cercas Computation and Language Artificial Intelligence Computers and Society Human-Computer Interaction Machine Learning Automated counter-narratives (CN) offer a promising strategy for mitigating online hate speech, yet concerns about their affective tone, accessibility, and ethical risks remain. We propose a framework for evaluating Large Language Model (LLM)-generated CNs across four dimensions: persona framing, verbosity and readability, affective tone, and ethical robustness. Using GPT-4o-Mini, Cohere's CommandR-7B, and Meta's LLaMA 3.1-70B, we assess three prompting strategies on the MT-Conan and HatEval datasets. Our findings reveal that LLM-generated CNs are often verbose and adapted for people with college-level literacy, limiting their accessibility. While emotionally guided prompts yield more empathetic and readable responses, there remain concerns surrounding safety and effectiveness. |
| title | Think Like a Person Before Responding: A Multi-Faceted Evaluation of Persona-Guided LLMs for Countering Hate |
| topic | Computation and Language Artificial Intelligence Computers and Society Human-Computer Interaction Machine Learning |
| url | https://arxiv.org/abs/2506.04043 |