AirGapAgent: Protecting Privacy-Conscious Conversational Agents
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909319328432128 |
|---|---|
| author | Bagdasarian, Eugene Yi, Ren Ghalebikesabi, Sahra Kairouz, Peter Gruteser, Marco Oh, Sewoong Balle, Borja Ramage, Daniel |
| author_facet | Bagdasarian, Eugene Yi, Ren Ghalebikesabi, Sahra Kairouz, Peter Gruteser, Marco Oh, Sewoong Balle, Borja Ramage, Daniel |
| contents | The growing use of large language model (LLM)-based conversational agents to manage sensitive user data raises significant privacy concerns. While these agents excel at understanding and acting on context, this capability can be exploited by malicious actors. We introduce a novel threat model where adversarial third-party apps manipulate the context of interaction to trick LLM-based agents into revealing private information not relevant to the task at hand.
Grounded in the framework of contextual integrity, we introduce AirGapAgent, a privacy-conscious agent designed to prevent unintended data leakage by restricting the agent's access to only the data necessary for a specific task. Extensive experiments using Gemini, GPT, and Mistral models as agents validate our approach's effectiveness in mitigating this form of context hijacking while maintaining core agent functionality. For example, we show that a single-query context hijacking attack on a Gemini Ultra agent reduces its ability to protect user data from 94% to 45%, while an AirGapAgent achieves 97% protection, rendering the same attack ineffective. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_05175 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AirGapAgent: Protecting Privacy-Conscious Conversational Agents Bagdasarian, Eugene Yi, Ren Ghalebikesabi, Sahra Kairouz, Peter Gruteser, Marco Oh, Sewoong Balle, Borja Ramage, Daniel Cryptography and Security Computation and Language Machine Learning The growing use of large language model (LLM)-based conversational agents to manage sensitive user data raises significant privacy concerns. While these agents excel at understanding and acting on context, this capability can be exploited by malicious actors. We introduce a novel threat model where adversarial third-party apps manipulate the context of interaction to trick LLM-based agents into revealing private information not relevant to the task at hand. Grounded in the framework of contextual integrity, we introduce AirGapAgent, a privacy-conscious agent designed to prevent unintended data leakage by restricting the agent's access to only the data necessary for a specific task. Extensive experiments using Gemini, GPT, and Mistral models as agents validate our approach's effectiveness in mitigating this form of context hijacking while maintaining core agent functionality. For example, we show that a single-query context hijacking attack on a Gemini Ultra agent reduces its ability to protect user data from 94% to 45%, while an AirGapAgent achieves 97% protection, rendering the same attack ineffective. |
| title | AirGapAgent: Protecting Privacy-Conscious Conversational Agents |
| topic | Cryptography and Security Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2405.05175 |