RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguards
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908608407535616 |
|---|---|
| author | Zheng, Jingnan Ji, Xiangtian Lu, Yijun Cui, Chenhang Zhao, Weixiang Deng, Gelei Liang, Zhenkai Zhang, An Chua, Tat-Seng |
| author_facet | Zheng, Jingnan Ji, Xiangtian Lu, Yijun Cui, Chenhang Zhao, Weixiang Deng, Gelei Liang, Zhenkai Zhang, An Chua, Tat-Seng |
| contents | Large Language Models (LLMs) continue to exhibit vulnerabilities despite deliberate safety alignment efforts, posing significant risks to users and society. To safeguard against the risk of policy-violating content, system-level moderation via external guard models-designed to monitor LLM inputs and outputs and block potentially harmful content-has emerged as a prevalent mitigation strategy. Existing approaches of training guard models rely heavily on extensive human curated datasets and struggle with out-of-distribution threats, such as emerging harmful categories or jailbreak attacks. To address these limitations, we propose RSafe, an adaptive reasoning-based safeguard that conducts guided safety reasoning to provide robust protection within the scope of specified safety policies. RSafe operates in two stages: 1) guided reasoning, where it analyzes safety risks of input content through policy-guided step-by-step reasoning, and 2) reinforced alignment, where rule-based RL optimizes its reasoning paths to align with accurate safety prediction. This two-stage training paradigm enables RSafe to internalize safety principles to generalize safety protection capability over unseen or adversarial safety violation scenarios. During inference, RSafe accepts user-specified safety policies to provide enhanced safeguards tailored to specific safety requirements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_07736 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguards Zheng, Jingnan Ji, Xiangtian Lu, Yijun Cui, Chenhang Zhao, Weixiang Deng, Gelei Liang, Zhenkai Zhang, An Chua, Tat-Seng Artificial Intelligence Large Language Models (LLMs) continue to exhibit vulnerabilities despite deliberate safety alignment efforts, posing significant risks to users and society. To safeguard against the risk of policy-violating content, system-level moderation via external guard models-designed to monitor LLM inputs and outputs and block potentially harmful content-has emerged as a prevalent mitigation strategy. Existing approaches of training guard models rely heavily on extensive human curated datasets and struggle with out-of-distribution threats, such as emerging harmful categories or jailbreak attacks. To address these limitations, we propose RSafe, an adaptive reasoning-based safeguard that conducts guided safety reasoning to provide robust protection within the scope of specified safety policies. RSafe operates in two stages: 1) guided reasoning, where it analyzes safety risks of input content through policy-guided step-by-step reasoning, and 2) reinforced alignment, where rule-based RL optimizes its reasoning paths to align with accurate safety prediction. This two-stage training paradigm enables RSafe to internalize safety principles to generalize safety protection capability over unseen or adversarial safety violation scenarios. During inference, RSafe accepts user-specified safety policies to provide enhanced safeguards tailored to specific safety requirements. |
| title | RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguards |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2506.07736 |