Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917708474351616 |
|---|---|
| author | Mireshghallah, Niloofar Kim, Hyunwoo Zhou, Xuhui Tsvetkov, Yulia Sap, Maarten Shokri, Reza Choi, Yejin |
| author_facet | Mireshghallah, Niloofar Kim, Hyunwoo Zhou, Xuhui Tsvetkov, Yulia Sap, Maarten Shokri, Reza Choi, Yejin |
| contents | The interactive use of large language models (LLMs) in AI assistants (at work, home, etc.) introduces a new set of inference-time privacy risks: LLMs are fed different types of information from multiple sources in their inputs and are expected to reason about what to share in their outputs, for what purpose and with whom, within a given context. In this work, we draw attention to the highly critical yet overlooked notion of contextual privacy by proposing ConfAIde, a benchmark designed to identify critical weaknesses in the privacy reasoning capabilities of instruction-tuned LLMs. Our experiments show that even the most capable models such as GPT-4 and ChatGPT reveal private information in contexts that humans would not, 39% and 57% of the time, respectively. This leakage persists even when we employ privacy-inducing prompts or chain-of-thought reasoning. Our work underscores the immediate need to explore novel inference-time privacy-preserving approaches, based on reasoning and theory of mind. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_17884 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory Mireshghallah, Niloofar Kim, Hyunwoo Zhou, Xuhui Tsvetkov, Yulia Sap, Maarten Shokri, Reza Choi, Yejin Artificial Intelligence Computation and Language Cryptography and Security The interactive use of large language models (LLMs) in AI assistants (at work, home, etc.) introduces a new set of inference-time privacy risks: LLMs are fed different types of information from multiple sources in their inputs and are expected to reason about what to share in their outputs, for what purpose and with whom, within a given context. In this work, we draw attention to the highly critical yet overlooked notion of contextual privacy by proposing ConfAIde, a benchmark designed to identify critical weaknesses in the privacy reasoning capabilities of instruction-tuned LLMs. Our experiments show that even the most capable models such as GPT-4 and ChatGPT reveal private information in contexts that humans would not, 39% and 57% of the time, respectively. This leakage persists even when we employ privacy-inducing prompts or chain-of-thought reasoning. Our work underscores the immediate need to explore novel inference-time privacy-preserving approaches, based on reasoning and theory of mind. |
| title | Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory |
| topic | Artificial Intelligence Computation and Language Cryptography and Security |
| url | https://arxiv.org/abs/2310.17884 |