Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917784161615872 |
|---|---|
| author | Rawat, Ambrish Schoepf, Stefan Zizzo, Giulio Cornacchia, Giandomenico Hameed, Muhammad Zaid Fraser, Kieran Miehling, Erik Buesser, Beat Daly, Elizabeth M. Purcell, Mark Sattigeri, Prasanna Chen, Pin-Yu Varshney, Kush R. |
| author_facet | Rawat, Ambrish Schoepf, Stefan Zizzo, Giulio Cornacchia, Giandomenico Hameed, Muhammad Zaid Fraser, Kieran Miehling, Erik Buesser, Beat Daly, Elizabeth M. Purcell, Mark Sattigeri, Prasanna Chen, Pin-Yu Varshney, Kush R. |
| contents | As generative AI, particularly large language models (LLMs), become increasingly integrated into production applications, new attack surfaces and vulnerabilities emerge and put a focus on adversarial threats in natural language and multi-modal systems. Red-teaming has gained importance in proactively identifying weaknesses in these systems, while blue-teaming works to protect against such adversarial attacks. Despite growing academic interest in adversarial risks for generative AI, there is limited guidance tailored for practitioners to assess and mitigate these challenges in real-world environments. To address this, our contributions include: (1) a practical examination of red- and blue-teaming strategies for securing generative AI, (2) identification of key challenges and open questions in defense development and evaluation, and (3) the Attack Atlas, an intuitive framework that brings a practical approach to analyzing single-turn input attacks, placing it at the forefront for practitioners. This work aims to bridge the gap between academic insights and practical security measures for the protection of generative AI systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_15398 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI Rawat, Ambrish Schoepf, Stefan Zizzo, Giulio Cornacchia, Giandomenico Hameed, Muhammad Zaid Fraser, Kieran Miehling, Erik Buesser, Beat Daly, Elizabeth M. Purcell, Mark Sattigeri, Prasanna Chen, Pin-Yu Varshney, Kush R. Cryptography and Security Artificial Intelligence Machine Learning As generative AI, particularly large language models (LLMs), become increasingly integrated into production applications, new attack surfaces and vulnerabilities emerge and put a focus on adversarial threats in natural language and multi-modal systems. Red-teaming has gained importance in proactively identifying weaknesses in these systems, while blue-teaming works to protect against such adversarial attacks. Despite growing academic interest in adversarial risks for generative AI, there is limited guidance tailored for practitioners to assess and mitigate these challenges in real-world environments. To address this, our contributions include: (1) a practical examination of red- and blue-teaming strategies for securing generative AI, (2) identification of key challenges and open questions in defense development and evaluation, and (3) the Attack Atlas, an intuitive framework that brings a practical approach to analyzing single-turn input attacks, placing it at the forefront for practitioners. This work aims to bridge the gap between academic insights and practical security measures for the protection of generative AI systems. |
| title | Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI |
| topic | Cryptography and Security Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2409.15398 |