Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.09974 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909493179187200 |
|---|---|
| author | Levin, Roman Cherepanova, Valeriia Hans, Abhimanyu Schwarzschild, Avi Goldstein, Tom |
| author_facet | Levin, Roman Cherepanova, Valeriia Hans, Abhimanyu Schwarzschild, Avi Goldstein, Tom |
| contents | Prompt engineering has emerged as a powerful technique for optimizing large language models (LLMs) for specific applications, enabling faster prototyping and improved performance, and giving rise to the interest of the community in protecting proprietary system prompts. In this work, we explore a novel perspective on prompt privacy through the lens of membership inference. We develop Prompt Detective, a statistical method to reliably determine whether a given system prompt was used by a third-party language model. Our approach relies on a statistical test comparing the distributions of two groups of model outputs corresponding to different system prompts. Through extensive experiments with a variety of language models, we demonstrate the effectiveness of Prompt Detective for prompt membership inference. Our work reveals that even minor changes in system prompts manifest in distinct response distributions, enabling us to verify prompt usage with statistical significance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_09974 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Has My System Prompt Been Used? Large Language Model Prompt Membership Inference Levin, Roman Cherepanova, Valeriia Hans, Abhimanyu Schwarzschild, Avi Goldstein, Tom Artificial Intelligence Cryptography and Security Prompt engineering has emerged as a powerful technique for optimizing large language models (LLMs) for specific applications, enabling faster prototyping and improved performance, and giving rise to the interest of the community in protecting proprietary system prompts. In this work, we explore a novel perspective on prompt privacy through the lens of membership inference. We develop Prompt Detective, a statistical method to reliably determine whether a given system prompt was used by a third-party language model. Our approach relies on a statistical test comparing the distributions of two groups of model outputs corresponding to different system prompts. Through extensive experiments with a variety of language models, we demonstrate the effectiveness of Prompt Detective for prompt membership inference. Our work reveals that even minor changes in system prompts manifest in distinct response distributions, enabling us to verify prompt usage with statistical significance. |
| title | Has My System Prompt Been Used? Large Language Model Prompt Membership Inference |
| topic | Artificial Intelligence Cryptography and Security |
| url | https://arxiv.org/abs/2502.09974 |