De-amplifying Bias from Differential Privacy in Language Model Fine-tuning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909096398028800 |
|---|---|
| author | Srivastava, Sanjari Mardziel, Piotr Zhang, Zhikhun Ahlawat, Archana Datta, Anupam Mitchell, John C |
| author_facet | Srivastava, Sanjari Mardziel, Piotr Zhang, Zhikhun Ahlawat, Archana Datta, Anupam Mitchell, John C |
| contents | Fairness and privacy are two important values machine learning (ML) practitioners often seek to operationalize in models. Fairness aims to reduce model bias for social/demographic sub-groups. Privacy via differential privacy (DP) mechanisms, on the other hand, limits the impact of any individual's training data on the resulting model. The trade-offs between privacy and fairness goals of trustworthy ML pose a challenge to those wishing to address both. We show that DP amplifies gender, racial, and religious bias when fine-tuning large language models (LLMs), producing models more biased than ones fine-tuned without DP. We find the cause of the amplification to be a disparity in convergence of gradients across sub-groups. Through the case of binary gender bias, we demonstrate that Counterfactual Data Augmentation (CDA), a known method for addressing bias, also mitigates bias amplification by DP. As a consequence, DP and CDA together can be used to fine-tune models while maintaining both fairness and privacy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_04489 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | De-amplifying Bias from Differential Privacy in Language Model Fine-tuning Srivastava, Sanjari Mardziel, Piotr Zhang, Zhikhun Ahlawat, Archana Datta, Anupam Mitchell, John C Machine Learning Cryptography and Security Computers and Society Methodology Fairness and privacy are two important values machine learning (ML) practitioners often seek to operationalize in models. Fairness aims to reduce model bias for social/demographic sub-groups. Privacy via differential privacy (DP) mechanisms, on the other hand, limits the impact of any individual's training data on the resulting model. The trade-offs between privacy and fairness goals of trustworthy ML pose a challenge to those wishing to address both. We show that DP amplifies gender, racial, and religious bias when fine-tuning large language models (LLMs), producing models more biased than ones fine-tuned without DP. We find the cause of the amplification to be a disparity in convergence of gradients across sub-groups. Through the case of binary gender bias, we demonstrate that Counterfactual Data Augmentation (CDA), a known method for addressing bias, also mitigates bias amplification by DP. As a consequence, DP and CDA together can be used to fine-tune models while maintaining both fairness and privacy. |
| title | De-amplifying Bias from Differential Privacy in Language Model Fine-tuning |
| topic | Machine Learning Cryptography and Security Computers and Society Methodology |
| url | https://arxiv.org/abs/2402.04489 |