Pr$εε$mpt: Sanitizing Sensitive Prompts for LLMs
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913992854732800 |
|---|---|
| author | Chowdhury, Amrita Roy Glukhov, David Anshumaan, Divyam Chalasani, Prasad Papernot, Nicolas Jha, Somesh Bellare, Mihir |
| author_facet | Chowdhury, Amrita Roy Glukhov, David Anshumaan, Divyam Chalasani, Prasad Papernot, Nicolas Jha, Somesh Bellare, Mihir |
| contents | The rise of large language models (LLMs) has introduced new privacy challenges, particularly during inference where sensitive information in prompts may be exposed to proprietary LLM APIs. In this paper, we address the problem of formally protecting the sensitive information contained in a prompt while maintaining response quality. To this end, first, we introduce a cryptographically inspired notion of a prompt sanitizer which transforms an input prompt to protect its sensitive tokens. Second, we propose Pr$εε$mpt, a novel system that implements a prompt sanitizer. Pr$εε$mpt categorizes sensitive tokens into two types: (1) those where the LLM's response depends solely on the format (such as SSNs, credit card numbers), for which we use format-preserving encryption (FPE); and (2) those where the response depends on specific values, (such as age, salary) for which we apply metric differential privacy (mDP). Our evaluation demonstrates that Pr$εε$mpt is a practical method to achieve meaningful privacy guarantees, while maintaining high utility compared to unsanitized prompts, and outperforming prior methods |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_05147 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Pr$εε$mpt: Sanitizing Sensitive Prompts for LLMs Chowdhury, Amrita Roy Glukhov, David Anshumaan, Divyam Chalasani, Prasad Papernot, Nicolas Jha, Somesh Bellare, Mihir Cryptography and Security Machine Learning The rise of large language models (LLMs) has introduced new privacy challenges, particularly during inference where sensitive information in prompts may be exposed to proprietary LLM APIs. In this paper, we address the problem of formally protecting the sensitive information contained in a prompt while maintaining response quality. To this end, first, we introduce a cryptographically inspired notion of a prompt sanitizer which transforms an input prompt to protect its sensitive tokens. Second, we propose Pr$εε$mpt, a novel system that implements a prompt sanitizer. Pr$εε$mpt categorizes sensitive tokens into two types: (1) those where the LLM's response depends solely on the format (such as SSNs, credit card numbers), for which we use format-preserving encryption (FPE); and (2) those where the response depends on specific values, (such as age, salary) for which we apply metric differential privacy (mDP). Our evaluation demonstrates that Pr$εε$mpt is a practical method to achieve meaningful privacy guarantees, while maintaining high utility compared to unsanitized prompts, and outperforming prior methods |
| title | Pr$εε$mpt: Sanitizing Sensitive Prompts for LLMs |
| topic | Cryptography and Security Machine Learning |
| url | https://arxiv.org/abs/2504.05147 |