Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2512.20061 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909974530097152 |
|---|---|
| author | Firooz, Hamed Liu, Rui Lu, Yuchen Hou, Zhenyu Xiong, Fangzhou Zhang, Xiaoyang Jian, Changshu Zhu, Zhicheng Ma, Jiayuan Tao, Jacob Gupta, Chaitali Peng, Xiaochang Mei, Shike Cui, Hang Qin, Yang Tang, Shuo Gaedtke, Jason Mittal, Arpit |
| author_facet | Firooz, Hamed Liu, Rui Lu, Yuchen Hou, Zhenyu Xiong, Fangzhou Zhang, Xiaoyang Jian, Changshu Zhu, Zhicheng Ma, Jiayuan Tao, Jacob Gupta, Chaitali Peng, Xiaochang Mei, Shike Cui, Hang Qin, Yang Tang, Shuo Gaedtke, Jason Mittal, Arpit |
| contents | Content moderation at scale remains one of the most pressing challenges in today's digital ecosystem, where billions of user- and AI-generated artifacts must be continuously evaluated for policy violations. Although recent advances in large language models (LLMs) have demonstrated strong potential for policy-grounded moderation, the practical challenges of training these systems to achieve expert-level accuracy in real-world settings remain largely unexplored, particularly in regimes characterized by label sparsity, evolving policy definitions, and the need for nuanced reasoning beyond shallow pattern matching. In this work, we present a comprehensive empirical investigation of scaling reinforcement learning (RL) for content classification, systematically evaluating multiple RL training recipes and reward-shaping strategies-including verifiable rewards and LLM-as-judge frameworks-to transform general-purpose language models into specialized, policy-aligned classifiers across three real-world content moderation tasks. Our findings provide actionable insights for industrial-scale moderation systems, demonstrating that RL exhibits sigmoid-like scaling behavior in which performance improves smoothly with increased training data, rollouts, and optimization steps before gradually saturating. Moreover, we show that RL substantially improves performance on tasks requiring complex policy-grounded reasoning while achieving up to 100x higher data efficiency than supervised fine-tuning, making it particularly effective in domains where expert annotations are scarce or costly. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_20061 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Scaling Reinforcement Learning for Content Moderation with Large Language Models Firooz, Hamed Liu, Rui Lu, Yuchen Hou, Zhenyu Xiong, Fangzhou Zhang, Xiaoyang Jian, Changshu Zhu, Zhicheng Ma, Jiayuan Tao, Jacob Gupta, Chaitali Peng, Xiaochang Mei, Shike Cui, Hang Qin, Yang Tang, Shuo Gaedtke, Jason Mittal, Arpit Artificial Intelligence Content moderation at scale remains one of the most pressing challenges in today's digital ecosystem, where billions of user- and AI-generated artifacts must be continuously evaluated for policy violations. Although recent advances in large language models (LLMs) have demonstrated strong potential for policy-grounded moderation, the practical challenges of training these systems to achieve expert-level accuracy in real-world settings remain largely unexplored, particularly in regimes characterized by label sparsity, evolving policy definitions, and the need for nuanced reasoning beyond shallow pattern matching. In this work, we present a comprehensive empirical investigation of scaling reinforcement learning (RL) for content classification, systematically evaluating multiple RL training recipes and reward-shaping strategies-including verifiable rewards and LLM-as-judge frameworks-to transform general-purpose language models into specialized, policy-aligned classifiers across three real-world content moderation tasks. Our findings provide actionable insights for industrial-scale moderation systems, demonstrating that RL exhibits sigmoid-like scaling behavior in which performance improves smoothly with increased training data, rollouts, and optimization steps before gradually saturating. Moreover, we show that RL substantially improves performance on tasks requiring complex policy-grounded reasoning while achieving up to 100x higher data efficiency than supervised fine-tuning, making it particularly effective in domains where expert annotations are scarce or costly. |
| title | Scaling Reinforcement Learning for Content Moderation with Large Language Models |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2512.20061 |