ShieldGemma: Generative AI Content Moderation Based on Gemma
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916345774342144 |
|---|---|
| author | Zeng, Wenjun Liu, Yuchi Mullins, Ryan Peran, Ludovic Fernandez, Joe Harkous, Hamza Narasimhan, Karthik Proud, Drew Kumar, Piyush Radharapu, Bhaktipriya Sturman, Olivia Wahltinez, Oscar |
| author_facet | Zeng, Wenjun Liu, Yuchi Mullins, Ryan Peran, Ludovic Fernandez, Joe Harkous, Hamza Narasimhan, Karthik Proud, Drew Kumar, Piyush Radharapu, Bhaktipriya Sturman, Olivia Wahltinez, Oscar |
| contents | We present ShieldGemma, a comprehensive suite of LLM-based safety content moderation models built upon Gemma2. These models provide robust, state-of-the-art predictions of safety risks across key harm types (sexually explicit, dangerous content, harassment, hate speech) in both user input and LLM-generated output. By evaluating on both public and internal benchmarks, we demonstrate superior performance compared to existing models, such as Llama Guard (+10.8\% AU-PRC on public benchmarks) and WildCard (+4.3\%). Additionally, we present a novel LLM-based data curation pipeline, adaptable to a variety of safety-related tasks and beyond. We have shown strong generalization performance for model trained mainly on synthetic data. By releasing ShieldGemma, we provide a valuable resource to the research community, advancing LLM safety and enabling the creation of more effective content moderation solutions for developers. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_21772 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ShieldGemma: Generative AI Content Moderation Based on Gemma Zeng, Wenjun Liu, Yuchi Mullins, Ryan Peran, Ludovic Fernandez, Joe Harkous, Hamza Narasimhan, Karthik Proud, Drew Kumar, Piyush Radharapu, Bhaktipriya Sturman, Olivia Wahltinez, Oscar Computation and Language Machine Learning We present ShieldGemma, a comprehensive suite of LLM-based safety content moderation models built upon Gemma2. These models provide robust, state-of-the-art predictions of safety risks across key harm types (sexually explicit, dangerous content, harassment, hate speech) in both user input and LLM-generated output. By evaluating on both public and internal benchmarks, we demonstrate superior performance compared to existing models, such as Llama Guard (+10.8\% AU-PRC on public benchmarks) and WildCard (+4.3\%). Additionally, we present a novel LLM-based data curation pipeline, adaptable to a variety of safety-related tasks and beyond. We have shown strong generalization performance for model trained mainly on synthetic data. By releasing ShieldGemma, we provide a valuable resource to the research community, advancing LLM safety and enabling the creation of more effective content moderation solutions for developers. |
| title | ShieldGemma: Generative AI Content Moderation Based on Gemma |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2407.21772 |