Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.12314 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917268575748096 |
|---|---|
| author | Xue, Zihao Bi, Zhen Ma, Long Hu, Zhenlin Wang, Yan Chen, Xueshu Liu, Zhenfang Zhao, Kang Xiao, Jie Lou, Jungang |
| author_facet | Xue, Zihao Bi, Zhen Ma, Long Hu, Zhenlin Wang, Yan Chen, Xueshu Liu, Zhenfang Zhao, Kang Xiao, Jie Lou, Jungang |
| contents | Large Reasoning Models (LRMs) leverage Chain-of-Thought (CoT) reasoning to solve complex tasks, but this explicit reasoning process introduces a critical vulnerability: adversarial manipulation of the thought chain itself, known as Chain-of-Thought Attacks (CoTA). Such attacks subtly corrupt the reasoning path to produce erroneous outputs, challenging conventional defenses that often sacrifice model utility for safety. To address this, we propose Thought Purity(TP), a defense framework that shifts from passive refusal to active reasoning recovery. TP integrates a safety-aware data pipeline with reinforcement learning, employing a dual-reward mechanism to teach models to dynamically identify and isolate malicious logic while preserving correct reasoning. Experiments on multiple model families demonstrate that TP significantly reduces the attack success rate of CoTA while maintaining or enhancing the model's performance on benign tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_12314 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Thought Purity: A Defense Framework For Chain-of-Thought Attack Xue, Zihao Bi, Zhen Ma, Long Hu, Zhenlin Wang, Yan Chen, Xueshu Liu, Zhenfang Zhao, Kang Xiao, Jie Lou, Jungang Machine Learning Artificial Intelligence Computational Engineering, Finance, and Science Cryptography and Security Large Reasoning Models (LRMs) leverage Chain-of-Thought (CoT) reasoning to solve complex tasks, but this explicit reasoning process introduces a critical vulnerability: adversarial manipulation of the thought chain itself, known as Chain-of-Thought Attacks (CoTA). Such attacks subtly corrupt the reasoning path to produce erroneous outputs, challenging conventional defenses that often sacrifice model utility for safety. To address this, we propose Thought Purity(TP), a defense framework that shifts from passive refusal to active reasoning recovery. TP integrates a safety-aware data pipeline with reinforcement learning, employing a dual-reward mechanism to teach models to dynamically identify and isolate malicious logic while preserving correct reasoning. Experiments on multiple model families demonstrate that TP significantly reduces the attack success rate of CoTA while maintaining or enhancing the model's performance on benign tasks. |
| title | Thought Purity: A Defense Framework For Chain-of-Thought Attack |
| topic | Machine Learning Artificial Intelligence Computational Engineering, Finance, and Science Cryptography and Security |
| url | https://arxiv.org/abs/2507.12314 |