A Multi-Perspective Benchmark and Moderation Model for Evaluating Safety and Adversarial Robustness
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910060456706048 |
|---|---|
| author | Machlovi, Naseem Saleki, Maryam Amin, Ruhul Rahouti, Mohamed Al-Maliki, Shawqi Qadir, Junaid Abdallah, Mohamed M. Al-Fuqaha, Ala |
| author_facet | Machlovi, Naseem Saleki, Maryam Amin, Ruhul Rahouti, Mohamed Al-Maliki, Shawqi Qadir, Junaid Abdallah, Mohamed M. Al-Fuqaha, Ala |
| contents | As large language models (LLMs) become deeply embedded in daily life, the urgent need for safer moderation systems that distinguish between naive and harmful requests while upholding appropriate censorship boundaries has never been greater. While existing LLMs can detect dangerous or unsafe content, they often struggle with nuanced cases such as implicit offensiveness, subtle gender and racial biases, and jailbreak prompts, due to the subjective and context-dependent nature of these issues. Furthermore, their heavy reliance on training data can reinforce societal biases, resulting in inconsistent and ethically problematic outputs. To address these challenges, we introduce GuardEval, a unified multi-perspective benchmark dataset designed for both training and evaluation, containing 106 fine-grained categories spanning human emotions, offensive and hateful language, gender and racial bias, and broader safety concerns. We also present GemmaGuard (GGuard), a Quantized Low-Rank Adaptation (QLoRA), fine-tuned version of Gemma3-12B trained on GuardEval, to assess content moderation with fine-grained labels. Our evaluation shows that GGuard achieves a macro F1 score of 0.832, substantially outperforming leading moderation models, including OpenAI Moderator (0.64) and Llama Guard (0.61). We show that multi-perspective, human-centered safety benchmarks are critical for mitigating inconsistent moderation decisions. GuardEval and GGuard together demonstrate that diverse, representative data materially improve safety, and adversarial robustness on complex, borderline cases. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_03273 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Multi-Perspective Benchmark and Moderation Model for Evaluating Safety and Adversarial Robustness Machlovi, Naseem Saleki, Maryam Amin, Ruhul Rahouti, Mohamed Al-Maliki, Shawqi Qadir, Junaid Abdallah, Mohamed M. Al-Fuqaha, Ala Computation and Language Artificial Intelligence Human-Computer Interaction Machine Learning I.2.7 As large language models (LLMs) become deeply embedded in daily life, the urgent need for safer moderation systems that distinguish between naive and harmful requests while upholding appropriate censorship boundaries has never been greater. While existing LLMs can detect dangerous or unsafe content, they often struggle with nuanced cases such as implicit offensiveness, subtle gender and racial biases, and jailbreak prompts, due to the subjective and context-dependent nature of these issues. Furthermore, their heavy reliance on training data can reinforce societal biases, resulting in inconsistent and ethically problematic outputs. To address these challenges, we introduce GuardEval, a unified multi-perspective benchmark dataset designed for both training and evaluation, containing 106 fine-grained categories spanning human emotions, offensive and hateful language, gender and racial bias, and broader safety concerns. We also present GemmaGuard (GGuard), a Quantized Low-Rank Adaptation (QLoRA), fine-tuned version of Gemma3-12B trained on GuardEval, to assess content moderation with fine-grained labels. Our evaluation shows that GGuard achieves a macro F1 score of 0.832, substantially outperforming leading moderation models, including OpenAI Moderator (0.64) and Llama Guard (0.61). We show that multi-perspective, human-centered safety benchmarks are critical for mitigating inconsistent moderation decisions. GuardEval and GGuard together demonstrate that diverse, representative data materially improve safety, and adversarial robustness on complex, borderline cases. |
| title | A Multi-Perspective Benchmark and Moderation Model for Evaluating Safety and Adversarial Robustness |
| topic | Computation and Language Artificial Intelligence Human-Computer Interaction Machine Learning I.2.7 |
| url | https://arxiv.org/abs/2601.03273 |