Prompt Injection attack against LLM-integrated Applications
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911341518782464 |
|---|---|
| author | Liu, Yi Deng, Gelei Li, Yuekang Wang, Kailong Wang, Zihao Wang, Xiaofeng Zhang, Tianwei Liu, Yepang Wang, Haoyu Zheng, Yan Zhang, Leo Yu Liu, Yang |
| author_facet | Liu, Yi Deng, Gelei Li, Yuekang Wang, Kailong Wang, Zihao Wang, Xiaofeng Zhang, Tianwei Liu, Yepang Wang, Haoyu Zheng, Yan Zhang, Leo Yu Liu, Yang |
| contents | Large Language Models (LLMs), renowned for their superior proficiency in language comprehension and generation, stimulate a vibrant ecosystem of applications around them. However, their extensive assimilation into various services introduces significant security risks. This study deconstructs the complexities and implications of prompt injection attacks on actual LLM-integrated applications. Initially, we conduct an exploratory analysis on ten commercial applications, highlighting the constraints of current attack strategies in practice. Prompted by these limitations, we subsequently formulate HouYi, a novel black-box prompt injection attack technique, which draws inspiration from traditional web injection attacks. HouYi is compartmentalized into three crucial elements: a seamlessly-incorporated pre-constructed prompt, an injection prompt inducing context partition, and a malicious payload designed to fulfill the attack objectives. Leveraging HouYi, we unveil previously unknown and severe attack outcomes, such as unrestricted arbitrary LLM usage and uncomplicated application prompt theft. We deploy HouYi on 36 actual LLM-integrated applications and discern 31 applications susceptible to prompt injection. 10 vendors have validated our discoveries, including Notion, which has the potential to impact millions of users. Our investigation illuminates both the possible risks of prompt injection attacks and the possible tactics for mitigation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_05499 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Prompt Injection attack against LLM-integrated Applications Liu, Yi Deng, Gelei Li, Yuekang Wang, Kailong Wang, Zihao Wang, Xiaofeng Zhang, Tianwei Liu, Yepang Wang, Haoyu Zheng, Yan Zhang, Leo Yu Liu, Yang Cryptography and Security Artificial Intelligence Computation and Language Software Engineering Large Language Models (LLMs), renowned for their superior proficiency in language comprehension and generation, stimulate a vibrant ecosystem of applications around them. However, their extensive assimilation into various services introduces significant security risks. This study deconstructs the complexities and implications of prompt injection attacks on actual LLM-integrated applications. Initially, we conduct an exploratory analysis on ten commercial applications, highlighting the constraints of current attack strategies in practice. Prompted by these limitations, we subsequently formulate HouYi, a novel black-box prompt injection attack technique, which draws inspiration from traditional web injection attacks. HouYi is compartmentalized into three crucial elements: a seamlessly-incorporated pre-constructed prompt, an injection prompt inducing context partition, and a malicious payload designed to fulfill the attack objectives. Leveraging HouYi, we unveil previously unknown and severe attack outcomes, such as unrestricted arbitrary LLM usage and uncomplicated application prompt theft. We deploy HouYi on 36 actual LLM-integrated applications and discern 31 applications susceptible to prompt injection. 10 vendors have validated our discoveries, including Notion, which has the potential to impact millions of users. Our investigation illuminates both the possible risks of prompt injection attacks and the possible tactics for mitigation. |
| title | Prompt Injection attack against LLM-integrated Applications |
| topic | Cryptography and Security Artificial Intelligence Computation and Language Software Engineering |
| url | https://arxiv.org/abs/2306.05499 |