Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.12659 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915621938135040 |
|---|---|
| author | Zhou, Kaiwen Liu, Chengzhi Zhao, Xuandong Jangam, Shreedhar Srinivasa, Jayanth Liu, Gaowen Song, Dawn Wang, Xin Eric |
| author_facet | Zhou, Kaiwen Liu, Chengzhi Zhao, Xuandong Jangam, Shreedhar Srinivasa, Jayanth Liu, Gaowen Song, Dawn Wang, Xin Eric |
| contents | The rapid development of large reasoning models (LRMs), such as OpenAI-o3 and DeepSeek-R1, has led to significant improvements in complex reasoning over non-reasoning large language models~(LLMs). However, their enhanced capabilities, combined with the open-source access of models like DeepSeek-R1, raise serious safety concerns, particularly regarding their potential for misuse. In this work, we present a comprehensive safety assessment of these reasoning models, leveraging established safety benchmarks to evaluate their compliance with safety regulations. Furthermore, we investigate their susceptibility to adversarial attacks, such as jailbreaking and prompt injection, to assess their robustness in real-world applications. Through our multi-faceted analysis, we uncover four key findings: (1) There is a significant safety gap between the open-source reasoning models and the o3-mini model, on both safety benchmark and attack, suggesting more safety effort on open LRMs is needed. (2) The stronger the model's reasoning ability, the greater the potential harm it may cause when answering unsafe questions. (3) Safety thinking emerges in the reasoning process of LRMs, but fails frequently against adversarial attacks. (4) The thinking process in R1 models poses greater safety concerns than their final answers. Our study provides insights into the security implications of reasoning models and highlights the need for further advancements in R1 models' safety to close the gap. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_12659 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1 Zhou, Kaiwen Liu, Chengzhi Zhao, Xuandong Jangam, Shreedhar Srinivasa, Jayanth Liu, Gaowen Song, Dawn Wang, Xin Eric Computers and Society Artificial Intelligence The rapid development of large reasoning models (LRMs), such as OpenAI-o3 and DeepSeek-R1, has led to significant improvements in complex reasoning over non-reasoning large language models~(LLMs). However, their enhanced capabilities, combined with the open-source access of models like DeepSeek-R1, raise serious safety concerns, particularly regarding their potential for misuse. In this work, we present a comprehensive safety assessment of these reasoning models, leveraging established safety benchmarks to evaluate their compliance with safety regulations. Furthermore, we investigate their susceptibility to adversarial attacks, such as jailbreaking and prompt injection, to assess their robustness in real-world applications. Through our multi-faceted analysis, we uncover four key findings: (1) There is a significant safety gap between the open-source reasoning models and the o3-mini model, on both safety benchmark and attack, suggesting more safety effort on open LRMs is needed. (2) The stronger the model's reasoning ability, the greater the potential harm it may cause when answering unsafe questions. (3) Safety thinking emerges in the reasoning process of LRMs, but fails frequently against adversarial attacks. (4) The thinking process in R1 models poses greater safety concerns than their final answers. Our study provides insights into the security implications of reasoning models and highlights the need for further advancements in R1 models' safety to close the gap. |
| title | The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1 |
| topic | Computers and Society Artificial Intelligence |
| url | https://arxiv.org/abs/2502.12659 |