Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908552188133376 |
|---|---|
| author | Wang, Shouju Yu, Fenglin Liu, Xirui Qin, Xiaoting Zhang, Jue Lin, Qingwei Zhang, Dongmei Rajmohan, Saravan |
| author_facet | Wang, Shouju Yu, Fenglin Liu, Xirui Qin, Xiaoting Zhang, Jue Lin, Qingwei Zhang, Dongmei Rajmohan, Saravan |
| contents | The increasing autonomy of LLM agents in handling sensitive communications, accelerated by Model Context Protocol (MCP) and Agent-to-Agent (A2A) frameworks, creates urgent privacy challenges. While recent work reveals significant gaps between LLMs' privacy Q&A performance and their agent behavior, existing benchmarks remain limited to static, simplified scenarios. We present PrivacyChecker, a model-agnostic, contextual integrity based mitigation approach that effectively reduces privacy leakage from 36.08% to 7.30% on DeepSeek-R1 and from 33.06% to 8.32% on GPT-4o, all while preserving task helpfulness. We also introduce PrivacyLens-Live, transforming static benchmarks into dynamic MCP and A2A environments that reveal substantially higher privacy risks in practical. Our modular mitigation approach integrates seamlessly into agent protocols through three deployment strategies, providing practical privacy protection for the emerging agentic ecosystem. Our data and code will be made available at https://aka.ms/privacy_in_action. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_17488 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents Wang, Shouju Yu, Fenglin Liu, Xirui Qin, Xiaoting Zhang, Jue Lin, Qingwei Zhang, Dongmei Rajmohan, Saravan Cryptography and Security Artificial Intelligence The increasing autonomy of LLM agents in handling sensitive communications, accelerated by Model Context Protocol (MCP) and Agent-to-Agent (A2A) frameworks, creates urgent privacy challenges. While recent work reveals significant gaps between LLMs' privacy Q&A performance and their agent behavior, existing benchmarks remain limited to static, simplified scenarios. We present PrivacyChecker, a model-agnostic, contextual integrity based mitigation approach that effectively reduces privacy leakage from 36.08% to 7.30% on DeepSeek-R1 and from 33.06% to 8.32% on GPT-4o, all while preserving task helpfulness. We also introduce PrivacyLens-Live, transforming static benchmarks into dynamic MCP and A2A environments that reveal substantially higher privacy risks in practical. Our modular mitigation approach integrates seamlessly into agent protocols through three deployment strategies, providing practical privacy protection for the emerging agentic ecosystem. Our data and code will be made available at https://aka.ms/privacy_in_action. |
| title | Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents |
| topic | Cryptography and Security Artificial Intelligence |
| url | https://arxiv.org/abs/2509.17488 |