Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2408.00925 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909277524852736 |
|---|---|
| author | Valbuena, Victor |
| author_facet | Valbuena, Victor |
| contents | The cross-prompt injection attack (XPIA) is an effective technique that can be used for data exfiltration, and that has seen increasing use. In this attack, the attacker injects a malicious instruction into third party data which an LLM is likely to consume when assisting a user, who is the victim. XPIA is often used as a means for data exfiltration, and the estimated cost of the average data breach for a business is nearly $4.5 million, which includes breaches such as compromised enterprise credentials. With the rise of gradient-based attacks such as the GCG suffix attack, the odds of an XPIA occurring which uses a GCG suffix are worryingly high. As part of my work in Microsoft's AI Red Team, I demonstrated a viable attack model using a GCG suffix paired with an injection in a simulated XPIA scenario. The results indicate that the presence of a GCG suffix can increase the odds of successful data exfiltration by nearly 20%, with some caveats. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_00925 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | WHITE PAPER: A Brief Exploration of Data Exfiltration using GCG Suffixes Valbuena, Victor Cryptography and Security Artificial Intelligence The cross-prompt injection attack (XPIA) is an effective technique that can be used for data exfiltration, and that has seen increasing use. In this attack, the attacker injects a malicious instruction into third party data which an LLM is likely to consume when assisting a user, who is the victim. XPIA is often used as a means for data exfiltration, and the estimated cost of the average data breach for a business is nearly $4.5 million, which includes breaches such as compromised enterprise credentials. With the rise of gradient-based attacks such as the GCG suffix attack, the odds of an XPIA occurring which uses a GCG suffix are worryingly high. As part of my work in Microsoft's AI Red Team, I demonstrated a viable attack model using a GCG suffix paired with an injection in a simulated XPIA scenario. The results indicate that the presence of a GCG suffix can increase the odds of successful data exfiltration by nearly 20%, with some caveats. |
| title | WHITE PAPER: A Brief Exploration of Data Exfiltration using GCG Suffixes |
| topic | Cryptography and Security Artificial Intelligence |
| url | https://arxiv.org/abs/2408.00925 |