Red Teaming for Generative AI, Report on a Copyright-Focused Exercise Completed in an Academic Medical Center
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918080713588736 |
|---|---|
| author | Wen, James Nalawade, Sahil Liang, Zhiwei Bielick, Catherine Boston, Marisa Ferrara Chowdhury, Alexander Collin, Adele De Angelis, Luigi Ellen, Jacob Frase, Heather Gameiro, Rodrigo R. Gutierrez, Juan Manuel Kadam, Pooja Keceli, Murat Krishnamurthy, Srikanth Kwok, Anne Lu, Yanan Lance Mattie, Heather McCoy, Liam G. Miller, Katherine Morgan, Allison C. Moerig, Marlene Louisa Nguyen, Trang Owen-Post, Alexander Ruiz, Alex D. Puchala, Sreekar Reddy Samineni, Soujanya Tohyama, Takeshi Ullanat, Varun Valenza, Carmine Velez, Camilo Wang, Pengcheng Wuest, Anna Zhou, Yuxiang Zhu, Yingde Johnson, Jason M. Lenane, Naomi Willcox, Jennifer Vitiello, Francis J. Celi, Leo Anthony G. Umeton, Renato |
| author_facet | Wen, James Nalawade, Sahil Liang, Zhiwei Bielick, Catherine Boston, Marisa Ferrara Chowdhury, Alexander Collin, Adele De Angelis, Luigi Ellen, Jacob Frase, Heather Gameiro, Rodrigo R. Gutierrez, Juan Manuel Kadam, Pooja Keceli, Murat Krishnamurthy, Srikanth Kwok, Anne Lu, Yanan Lance Mattie, Heather McCoy, Liam G. Miller, Katherine Morgan, Allison C. Moerig, Marlene Louisa Nguyen, Trang Owen-Post, Alexander Ruiz, Alex D. Puchala, Sreekar Reddy Samineni, Soujanya Tohyama, Takeshi Ullanat, Varun Valenza, Carmine Velez, Camilo Wang, Pengcheng Wuest, Anna Zhou, Yuxiang Zhu, Yingde Johnson, Jason M. Lenane, Naomi Willcox, Jennifer Vitiello, Francis J. Celi, Leo Anthony G. Umeton, Renato |
| contents | Background: Generative artificial intelligence (AI) deployment in academic medical settings raises copyright compliance concerns. Dana-Farber Cancer Institute implemented GPT4DFCI, an internal generative AI tool utilizing OpenAI models, that is approved for enterprise use in research and operations. Given (1) the exceptionally broad adoption of the tool in our organization, (2) our research mission, and (3) the shared responsibility model required to benefit from Customer Copyright Commitment in Azure OpenAI Service products, we deemed rigorous copyright compliance testing necessary.
Case Description: We conducted a structured red teaming exercise in Nov. 2024, with 42 participants from academic, industry, and government institutions. Four teams attempted to extract copyrighted content from GPT4DFCI across four domains: literary works, news articles, scientific publications, and access-restricted clinical notes. Teams successfully extracted verbatim book dedications and near-exact passages through various strategies. News article extraction failed despite jailbreak attempts. Scientific article reproduction yielded only high-level summaries. Clinical note testing revealed appropriate privacy safeguards.
Discussion: The successful extraction of literary content indicates potential copyrighted material presence in training data, necessitating inference-time filtering. Differential success rates across content types suggest varying protective mechanisms. The event led to implementation of a copyright-specific meta-prompt in GPT4DFCI; this mitigation has been in production since Jan. 2025.
Conclusion: Systematic red teaming revealed specific vulnerabilities in generative AI copyright compliance, leading to concrete mitigation strategies. Academic medical institutions deploying generative AI should implement continuous testing protocols to ensure legal and ethical compliance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_22523 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Red Teaming for Generative AI, Report on a Copyright-Focused Exercise Completed in an Academic Medical Center Wen, James Nalawade, Sahil Liang, Zhiwei Bielick, Catherine Boston, Marisa Ferrara Chowdhury, Alexander Collin, Adele De Angelis, Luigi Ellen, Jacob Frase, Heather Gameiro, Rodrigo R. Gutierrez, Juan Manuel Kadam, Pooja Keceli, Murat Krishnamurthy, Srikanth Kwok, Anne Lu, Yanan Lance Mattie, Heather McCoy, Liam G. Miller, Katherine Morgan, Allison C. Moerig, Marlene Louisa Nguyen, Trang Owen-Post, Alexander Ruiz, Alex D. Puchala, Sreekar Reddy Samineni, Soujanya Tohyama, Takeshi Ullanat, Varun Valenza, Carmine Velez, Camilo Wang, Pengcheng Wuest, Anna Zhou, Yuxiang Zhu, Yingde Johnson, Jason M. Lenane, Naomi Willcox, Jennifer Vitiello, Francis J. Celi, Leo Anthony G. Umeton, Renato Computers and Society Artificial Intelligence Background: Generative artificial intelligence (AI) deployment in academic medical settings raises copyright compliance concerns. Dana-Farber Cancer Institute implemented GPT4DFCI, an internal generative AI tool utilizing OpenAI models, that is approved for enterprise use in research and operations. Given (1) the exceptionally broad adoption of the tool in our organization, (2) our research mission, and (3) the shared responsibility model required to benefit from Customer Copyright Commitment in Azure OpenAI Service products, we deemed rigorous copyright compliance testing necessary. Case Description: We conducted a structured red teaming exercise in Nov. 2024, with 42 participants from academic, industry, and government institutions. Four teams attempted to extract copyrighted content from GPT4DFCI across four domains: literary works, news articles, scientific publications, and access-restricted clinical notes. Teams successfully extracted verbatim book dedications and near-exact passages through various strategies. News article extraction failed despite jailbreak attempts. Scientific article reproduction yielded only high-level summaries. Clinical note testing revealed appropriate privacy safeguards. Discussion: The successful extraction of literary content indicates potential copyrighted material presence in training data, necessitating inference-time filtering. Differential success rates across content types suggest varying protective mechanisms. The event led to implementation of a copyright-specific meta-prompt in GPT4DFCI; this mitigation has been in production since Jan. 2025. Conclusion: Systematic red teaming revealed specific vulnerabilities in generative AI copyright compliance, leading to concrete mitigation strategies. Academic medical institutions deploying generative AI should implement continuous testing protocols to ensure legal and ethical compliance. |
| title | Red Teaming for Generative AI, Report on a Copyright-Focused Exercise Completed in an Academic Medical Center |
| topic | Computers and Society Artificial Intelligence |
| url | https://arxiv.org/abs/2506.22523 |