Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.23576 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911029743583232 |
|---|---|
| author | Wit, Maria Carolina Cornelia Pang, Jun |
| author_facet | Wit, Maria Carolina Cornelia Pang, Jun |
| contents | Recent advances in large language models (LLMs) have raised concerns about jailbreaking attacks, i.e., prompts that bypass safety mechanisms. This paper investigates the use of multi-agent LLM systems as a defence against such attacks. We evaluate three jailbreaking strategies, including the original AutoDefense attack and two from Deepleaps: BetterDan and JB. Reproducing the AutoDefense framework, we compare single-agent setups with two- and three-agent configurations. Our results show that multi-agent systems enhance resistance to jailbreaks, especially by reducing false negatives. However, its effectiveness varies by attack type, and it introduces trade-offs such as increased false positives and computational overhead. These findings point to the limitations of current automated defences and suggest directions for improving alignment robustness in future LLM systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_23576 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating Multi-Agent Defences Against Jailbreaking Attacks on Large Language Models Wit, Maria Carolina Cornelia Pang, Jun Artificial Intelligence Recent advances in large language models (LLMs) have raised concerns about jailbreaking attacks, i.e., prompts that bypass safety mechanisms. This paper investigates the use of multi-agent LLM systems as a defence against such attacks. We evaluate three jailbreaking strategies, including the original AutoDefense attack and two from Deepleaps: BetterDan and JB. Reproducing the AutoDefense framework, we compare single-agent setups with two- and three-agent configurations. Our results show that multi-agent systems enhance resistance to jailbreaks, especially by reducing false negatives. However, its effectiveness varies by attack type, and it introduces trade-offs such as increased false positives and computational overhead. These findings point to the limitations of current automated defences and suggest directions for improving alignment robustness in future LLM systems. |
| title | Evaluating Multi-Agent Defences Against Jailbreaking Attacks on Large Language Models |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2506.23576 |