Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910176817184768 |
|---|---|
| author | de Witt, Christian Schroeder Krawiecka, Klaudia Krawczuk, Igor Hagag, Ben Anderson, William L. Belcak, Peter Bucknall, Ben Cai, Xiaohong Chopra, Ayush Cohen, Doron Del Rosario, Ron F. Draguns, Andis Gray, Annie Katz, Keren Mavroudis, Vasilios Mink, Jaron Motwani, Sumeet Ramesh Petit, Jonathan Rembeck, Leif-Sebastian Smith, Chandler Sotiropoulos, John Young, Steven Scheffler, Sarah Llewellyn, Mary |
| author_facet | de Witt, Christian Schroeder Krawiecka, Klaudia Krawczuk, Igor Hagag, Ben Anderson, William L. Belcak, Peter Bucknall, Ben Cai, Xiaohong Chopra, Ayush Cohen, Doron Del Rosario, Ron F. Draguns, Andis Gray, Annie Katz, Keren Mavroudis, Vasilios Mink, Jaron Motwani, Sumeet Ramesh Petit, Jonathan Rembeck, Leif-Sebastian Smith, Chandler Sotiropoulos, John Young, Steven Scheffler, Sarah Llewellyn, Mary |
| contents | AI agents are beginning to interact with each other directly and across internet platforms and physical environments, creating security challenges beyond traditional cybersecurity and AI safety frameworks. Free-form protocols are essential for AI's task generalization but enable new threats like secret collusion and coordinated swarm attacks. Network effects can rapidly spread privacy breaches, disinformation, jailbreaks, and data poisoning, while multi-agent dispersion and stealth optimization help adversaries evade oversight - creating novel persistent threats at a systemic level. Despite their critical importance, these security challenges remain understudied, with research fragmented across disparate fields including AI security, multi-agent learning, complex systems, cybersecurity, game theory, distributed systems, and technical AI governance. We introduce multi-agent security, a new field dedicated to securing networks of AI agents against threats that emerge or amplify through their interactions - whether direct or indirect via shared environments - with each other, humans, and institutions, and characterise fundamental security-utility and security-security trade-offs across both distributed and decentralised settings. Our preliminary work (1) taxonomizes the threat landscape arising from interacting AI agents, (2) offers applications to multi-agent security for work across diffuse subfields, and (3) proposes a unified research agenda addressing open challenges in designing secure agent systems and interaction environments. By identifying these gaps, we aim to guide research in this critical area to unlock the socioeconomic potential of large-scale agent deployment, foster public trust, and mitigate national security risks in critical infrastructure and defense contexts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_02077 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents de Witt, Christian Schroeder Krawiecka, Klaudia Krawczuk, Igor Hagag, Ben Anderson, William L. Belcak, Peter Bucknall, Ben Cai, Xiaohong Chopra, Ayush Cohen, Doron Del Rosario, Ron F. Draguns, Andis Gray, Annie Katz, Keren Mavroudis, Vasilios Mink, Jaron Motwani, Sumeet Ramesh Petit, Jonathan Rembeck, Leif-Sebastian Smith, Chandler Sotiropoulos, John Young, Steven Scheffler, Sarah Llewellyn, Mary Cryptography and Security Artificial Intelligence Multiagent Systems AI agents are beginning to interact with each other directly and across internet platforms and physical environments, creating security challenges beyond traditional cybersecurity and AI safety frameworks. Free-form protocols are essential for AI's task generalization but enable new threats like secret collusion and coordinated swarm attacks. Network effects can rapidly spread privacy breaches, disinformation, jailbreaks, and data poisoning, while multi-agent dispersion and stealth optimization help adversaries evade oversight - creating novel persistent threats at a systemic level. Despite their critical importance, these security challenges remain understudied, with research fragmented across disparate fields including AI security, multi-agent learning, complex systems, cybersecurity, game theory, distributed systems, and technical AI governance. We introduce multi-agent security, a new field dedicated to securing networks of AI agents against threats that emerge or amplify through their interactions - whether direct or indirect via shared environments - with each other, humans, and institutions, and characterise fundamental security-utility and security-security trade-offs across both distributed and decentralised settings. Our preliminary work (1) taxonomizes the threat landscape arising from interacting AI agents, (2) offers applications to multi-agent security for work across diffuse subfields, and (3) proposes a unified research agenda addressing open challenges in designing secure agent systems and interaction environments. By identifying these gaps, we aim to guide research in this critical area to unlock the socioeconomic potential of large-scale agent deployment, foster public trust, and mitigate national security risks in critical infrastructure and defense contexts. |
| title | Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents |
| topic | Cryptography and Security Artificial Intelligence Multiagent Systems |
| url | https://arxiv.org/abs/2505.02077 |