fairBERTs: Erasing Sensitive Information Through Semantic and Fairness-aware Perturbations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911952690741248 |
|---|---|
| author | Li, Jinfeng Chen, Yuefeng Liu, Xiangyu Huang, Longtao Zhang, Rong Xue, Hui |
| author_facet | Li, Jinfeng Chen, Yuefeng Liu, Xiangyu Huang, Longtao Zhang, Rong Xue, Hui |
| contents | Pre-trained language models (PLMs) have revolutionized both the natural language processing research and applications. However, stereotypical biases (e.g., gender and racial discrimination) encoded in PLMs have raised negative ethical implications for PLMs, which critically limits their broader applications. To address the aforementioned unfairness issues, we present fairBERTs, a general framework for learning fair fine-tuned BERT series models by erasing the protected sensitive information via semantic and fairness-aware perturbations generated by a generative adversarial network. Through extensive qualitative and quantitative experiments on two real-world tasks, we demonstrate the great superiority of fairBERTs in mitigating unfairness while maintaining the model utility. We also verify the feasibility of transferring adversarial components in fairBERTs to other conventionally trained BERT-like models for yielding fairness improvements. Our findings may shed light on further research on building fairer fine-tuned PLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_08189 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | fairBERTs: Erasing Sensitive Information Through Semantic and Fairness-aware Perturbations Li, Jinfeng Chen, Yuefeng Liu, Xiangyu Huang, Longtao Zhang, Rong Xue, Hui Computation and Language Artificial Intelligence Pre-trained language models (PLMs) have revolutionized both the natural language processing research and applications. However, stereotypical biases (e.g., gender and racial discrimination) encoded in PLMs have raised negative ethical implications for PLMs, which critically limits their broader applications. To address the aforementioned unfairness issues, we present fairBERTs, a general framework for learning fair fine-tuned BERT series models by erasing the protected sensitive information via semantic and fairness-aware perturbations generated by a generative adversarial network. Through extensive qualitative and quantitative experiments on two real-world tasks, we demonstrate the great superiority of fairBERTs in mitigating unfairness while maintaining the model utility. We also verify the feasibility of transferring adversarial components in fairBERTs to other conventionally trained BERT-like models for yielding fairness improvements. Our findings may shed light on further research on building fairer fine-tuned PLMs. |
| title | fairBERTs: Erasing Sensitive Information Through Semantic and Fairness-aware Perturbations |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2407.08189 |