Class-RAG: Real-Time Content Moderation with Retrieval Augmented Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910750483677184 |
|---|---|
| author | Chen, Jianfa Shen, Emily Bavalatti, Trupti Lin, Xiaowen Wang, Yongkai Hu, Shuming Subramanyam, Harihar Vepuri, Ksheeraj Sai Jiang, Ming Qi, Ji Chen, Li Jiang, Nan Jain, Ankit |
| author_facet | Chen, Jianfa Shen, Emily Bavalatti, Trupti Lin, Xiaowen Wang, Yongkai Hu, Shuming Subramanyam, Harihar Vepuri, Ksheeraj Sai Jiang, Ming Qi, Ji Chen, Li Jiang, Nan Jain, Ankit |
| contents | Robust content moderation classifiers are essential for the safety of Generative AI systems. In this task, differences between safe and unsafe inputs are often extremely subtle, making it difficult for classifiers (and indeed, even humans) to properly distinguish violating vs. benign samples without context or explanation. Scaling risk discovery and mitigation through continuous model fine-tuning is also slow, challenging and costly, preventing developers from being able to respond quickly and effectively to emergent harms. We propose a Classification approach employing Retrieval-Augmented Generation (Class-RAG). Class-RAG extends the capability of its base LLM through access to a retrieval library which can be dynamically updated to enable semantic hotfixing for immediate, flexible risk mitigation. Compared to model fine-tuning, Class-RAG demonstrates flexibility and transparency in decision-making, outperforms on classification and is more robust against adversarial attack, as evidenced by empirical studies. Our findings also suggest that Class-RAG performance scales with retrieval library size, indicating that increasing the library size is a viable and low-cost approach to improve content moderation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_14881 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Class-RAG: Real-Time Content Moderation with Retrieval Augmented Generation Chen, Jianfa Shen, Emily Bavalatti, Trupti Lin, Xiaowen Wang, Yongkai Hu, Shuming Subramanyam, Harihar Vepuri, Ksheeraj Sai Jiang, Ming Qi, Ji Chen, Li Jiang, Nan Jain, Ankit Artificial Intelligence Computation and Language Robust content moderation classifiers are essential for the safety of Generative AI systems. In this task, differences between safe and unsafe inputs are often extremely subtle, making it difficult for classifiers (and indeed, even humans) to properly distinguish violating vs. benign samples without context or explanation. Scaling risk discovery and mitigation through continuous model fine-tuning is also slow, challenging and costly, preventing developers from being able to respond quickly and effectively to emergent harms. We propose a Classification approach employing Retrieval-Augmented Generation (Class-RAG). Class-RAG extends the capability of its base LLM through access to a retrieval library which can be dynamically updated to enable semantic hotfixing for immediate, flexible risk mitigation. Compared to model fine-tuning, Class-RAG demonstrates flexibility and transparency in decision-making, outperforms on classification and is more robust against adversarial attack, as evidenced by empirical studies. Our findings also suggest that Class-RAG performance scales with retrieval library size, indicating that increasing the library size is a viable and low-cost approach to improve content moderation. |
| title | Class-RAG: Real-Time Content Moderation with Retrieval Augmented Generation |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2410.14881 |