Stepwise Reasoning Error Disruption Attack of LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908407548608512 |
|---|---|
| author | Peng, Jingyu Wang, Maolin Zhao, Xiangyu Zhang, Kai Wang, Wanyu Jia, Pengyue Liu, Qidong Guo, Ruocheng Liu, Qi |
| author_facet | Peng, Jingyu Wang, Maolin Zhao, Xiangyu Zhang, Kai Wang, Wanyu Jia, Pengyue Liu, Qidong Guo, Ruocheng Liu, Qi |
| contents | Large language models (LLMs) have made remarkable strides in complex reasoning tasks, but their safety and robustness in reasoning processes remain underexplored. Existing attacks on LLM reasoning are constrained by specific settings or lack of imperceptibility, limiting their feasibility and generalizability. To address these challenges, we propose the Stepwise rEasoning Error Disruption (SEED) attack, which subtly injects errors into prior reasoning steps to mislead the model into producing incorrect subsequent reasoning and final answers. Unlike previous methods, SEED is compatible with zero-shot and few-shot settings, maintains the natural reasoning flow, and ensures covert execution without modifying the instruction. Extensive experiments on four datasets across four different models demonstrate SEED's effectiveness, revealing the vulnerabilities of LLMs to disruptions in reasoning processes. These findings underscore the need for greater attention to the robustness of LLM reasoning to ensure safety in practical applications. Our code is available at: https://github.com/Applied-Machine-Learning-Lab/SEED-Attack. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_11934 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Stepwise Reasoning Error Disruption Attack of LLMs Peng, Jingyu Wang, Maolin Zhao, Xiangyu Zhang, Kai Wang, Wanyu Jia, Pengyue Liu, Qidong Guo, Ruocheng Liu, Qi Artificial Intelligence Large language models (LLMs) have made remarkable strides in complex reasoning tasks, but their safety and robustness in reasoning processes remain underexplored. Existing attacks on LLM reasoning are constrained by specific settings or lack of imperceptibility, limiting their feasibility and generalizability. To address these challenges, we propose the Stepwise rEasoning Error Disruption (SEED) attack, which subtly injects errors into prior reasoning steps to mislead the model into producing incorrect subsequent reasoning and final answers. Unlike previous methods, SEED is compatible with zero-shot and few-shot settings, maintains the natural reasoning flow, and ensures covert execution without modifying the instruction. Extensive experiments on four datasets across four different models demonstrate SEED's effectiveness, revealing the vulnerabilities of LLMs to disruptions in reasoning processes. These findings underscore the need for greater attention to the robustness of LLM reasoning to ensure safety in practical applications. Our code is available at: https://github.com/Applied-Machine-Learning-Lab/SEED-Attack. |
| title | Stepwise Reasoning Error Disruption Attack of LLMs |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2412.11934 |