Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.04415 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912145886674944 |
|---|---|
| author | Li, Xuying Li, Zhuo Kosuga, Yuji Yoshida, Yasuhiro Bian, Victor |
| author_facet | Li, Xuying Li, Zhuo Kosuga, Yuji Yoshida, Yasuhiro Bian, Victor |
| contents | AI agents, powered by large language models (LLMs), have transformed human-computer interactions by enabling seamless, natural, and context-aware communication. While these advancements offer immense utility, they also inherit and amplify inherent safety risks such as bias, fairness, hallucinations, privacy breaches, and a lack of transparency. This paper investigates a critical vulnerability: adversarial attacks targeting the LLM core within AI agents. Specifically, we test the hypothesis that a deceptively simple adversarial prefix, such as \textit{Ignore the document}, can compel LLMs to produce dangerous or unintended outputs by bypassing their contextual safeguards. Through experimentation, we demonstrate a high attack success rate (ASR), revealing the fragility of existing LLM defenses. These findings emphasize the urgent need for robust, multi-layered security measures tailored to mitigate vulnerabilities at the LLM level and within broader agent-based architectures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_04415 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Targeting the Core: A Simple and Effective Method to Attack RAG-based Agents via Direct LLM Manipulation Li, Xuying Li, Zhuo Kosuga, Yuji Yoshida, Yasuhiro Bian, Victor Artificial Intelligence AI agents, powered by large language models (LLMs), have transformed human-computer interactions by enabling seamless, natural, and context-aware communication. While these advancements offer immense utility, they also inherit and amplify inherent safety risks such as bias, fairness, hallucinations, privacy breaches, and a lack of transparency. This paper investigates a critical vulnerability: adversarial attacks targeting the LLM core within AI agents. Specifically, we test the hypothesis that a deceptively simple adversarial prefix, such as \textit{Ignore the document}, can compel LLMs to produce dangerous or unintended outputs by bypassing their contextual safeguards. Through experimentation, we demonstrate a high attack success rate (ASR), revealing the fragility of existing LLM defenses. These findings emphasize the urgent need for robust, multi-layered security measures tailored to mitigate vulnerabilities at the LLM level and within broader agent-based architectures. |
| title | Targeting the Core: A Simple and Effective Method to Attack RAG-based Agents via Direct LLM Manipulation |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2412.04415 |