A methodological analysis of prompt perturbations and their effect on attack success rates
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912707517612032 |
|---|---|
| author | Machado, Tiago de Macedo, Maysa Malfiza Garcia de Paula, Rogerio Abreu Grave, Marcelo Carpinette Adebiyi, Aminat de Souza, Luan Soares Santarelli, Enrico Pinhanez, Claudio |
| author_facet | Machado, Tiago de Macedo, Maysa Malfiza Garcia de Paula, Rogerio Abreu Grave, Marcelo Carpinette Adebiyi, Aminat de Souza, Luan Soares Santarelli, Enrico Pinhanez, Claudio |
| contents | This work aims to investigate how different Large Language Models (LLMs) alignment methods affect the models' responses to prompt attacks. We selected open source models based on the most common alignment methods, namely, Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Reinforcement Learning with Human Feedback (RLHF). We conducted a systematic analysis using statistical methods to verify how sensitive the Attack Success Rate (ASR) is when we apply variations to prompts designed to elicit inappropriate content from LLMs. Our results show that even small prompt modifications can significantly change the Attack Success Rate (ASR) according to the statistical tests we run, making the models more or less susceptible to types of attack. Critically, our results demonstrate that running existing 'attack benchmarks' alone may not be sufficient to elicit all possible vulnerabilities of both models and alignment methods. This paper thus contributes to ongoing efforts on model attack evaluation by means of systematic and statistically-based analyses of the different alignment methods and how sensitive their ASR is to prompt variation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_10686 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A methodological analysis of prompt perturbations and their effect on attack success rates Machado, Tiago de Macedo, Maysa Malfiza Garcia de Paula, Rogerio Abreu Grave, Marcelo Carpinette Adebiyi, Aminat de Souza, Luan Soares Santarelli, Enrico Pinhanez, Claudio Computation and Language 62, 68, I.2.7 This work aims to investigate how different Large Language Models (LLMs) alignment methods affect the models' responses to prompt attacks. We selected open source models based on the most common alignment methods, namely, Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Reinforcement Learning with Human Feedback (RLHF). We conducted a systematic analysis using statistical methods to verify how sensitive the Attack Success Rate (ASR) is when we apply variations to prompts designed to elicit inappropriate content from LLMs. Our results show that even small prompt modifications can significantly change the Attack Success Rate (ASR) according to the statistical tests we run, making the models more or less susceptible to types of attack. Critically, our results demonstrate that running existing 'attack benchmarks' alone may not be sufficient to elicit all possible vulnerabilities of both models and alignment methods. This paper thus contributes to ongoing efforts on model attack evaluation by means of systematic and statistically-based analyses of the different alignment methods and how sensitive their ASR is to prompt variation. |
| title | A methodological analysis of prompt perturbations and their effect on attack success rates |
| topic | Computation and Language 62, 68, I.2.7 |
| url | https://arxiv.org/abs/2511.10686 |