Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2307.16382 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913316306157568 |
|---|---|
| author | Sun, Albert Yu Zemour, Eliott Saxena, Arushi Vaidyanathan, Udith Lin, Eric Lau, Christian Mugunthan, Vaikkunth |
| author_facet | Sun, Albert Yu Zemour, Eliott Saxena, Arushi Vaidyanathan, Udith Lin, Eric Lau, Christian Mugunthan, Vaikkunth |
| contents | Machine learning practitioners often fine-tune generative pre-trained models like GPT-3 to improve model performance at specific tasks. Previous works, however, suggest that fine-tuned machine learning models memorize and emit sensitive information from the original fine-tuning dataset. Companies such as OpenAI offer fine-tuning services for their models, but no prior work has conducted a memorization attack on any closed-source models. In this work, we simulate a privacy attack on GPT-3 using OpenAI's fine-tuning API. Our objective is to determine if personally identifiable information (PII) can be extracted from this model. We (1) explore the use of naive prompting methods on a GPT-3 fine-tuned classification model, and (2) we design a practical word generation task called Autocomplete to investigate the extent of PII memorization in fine-tuned GPT-3 within a real-world context. Our findings reveal that fine-tuning GPT3 for both tasks led to the model memorizing and disclosing critical personally identifiable information (PII) obtained from the underlying fine-tuning dataset. To encourage further research, we have made our codes and datasets publicly available on GitHub at: https://github.com/albertsun1/gpt3-pii-attacks |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2307_16382 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Does fine-tuning GPT-3 with the OpenAI API leak personally-identifiable information? Sun, Albert Yu Zemour, Eliott Saxena, Arushi Vaidyanathan, Udith Lin, Eric Lau, Christian Mugunthan, Vaikkunth Machine Learning Computation and Language Machine learning practitioners often fine-tune generative pre-trained models like GPT-3 to improve model performance at specific tasks. Previous works, however, suggest that fine-tuned machine learning models memorize and emit sensitive information from the original fine-tuning dataset. Companies such as OpenAI offer fine-tuning services for their models, but no prior work has conducted a memorization attack on any closed-source models. In this work, we simulate a privacy attack on GPT-3 using OpenAI's fine-tuning API. Our objective is to determine if personally identifiable information (PII) can be extracted from this model. We (1) explore the use of naive prompting methods on a GPT-3 fine-tuned classification model, and (2) we design a practical word generation task called Autocomplete to investigate the extent of PII memorization in fine-tuned GPT-3 within a real-world context. Our findings reveal that fine-tuning GPT3 for both tasks led to the model memorizing and disclosing critical personally identifiable information (PII) obtained from the underlying fine-tuning dataset. To encourage further research, we have made our codes and datasets publicly available on GitHub at: https://github.com/albertsun1/gpt3-pii-attacks |
| title | Does fine-tuning GPT-3 with the OpenAI API leak personally-identifiable information? |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2307.16382 |