Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.16780 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913042165399552 |
|---|---|
| author | Tang, Qiaoyue Hosseini, Sepidehsadat Zhai, Mengyao Durand, Thibaut Mori, Greg |
| author_facet | Tang, Qiaoyue Hosseini, Sepidehsadat Zhai, Mengyao Durand, Thibaut Mori, Greg |
| contents | This paper presents FairNVT, a lightweight debiasing framework for pretrained transformer-based encoders that improves both representation and prediction level fairness while preserving task accuracy. Unlike many existing debiasing approaches that address these notions separately, we argue they are inherently connected: suppressing sensitive information at the representation level can facilitate fairer predictions. Our approach learns task-relevant and sensitive embeddings via lightweight adapters, applies calibrated Gaussian noise to the sensitive embedding, and fuses it with the task representation. Together with orthogonality constraints and fairness regularization, these components jointly reduce sensitive-attribute leakage in the learned embeddings and encourage fairer downstream predictions. The framework is compatible with a wide range of pretrained transformer encoders. Across three datasets spanning vision and language, FairNVT reduces sensitive-attribute attacker accuracy, improves demographic-parity and equalized-odds metrics, and maintains high task performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_16780 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | FairNVT: Improving Fairness via Noise Injection in Vision Transformers Tang, Qiaoyue Hosseini, Sepidehsadat Zhai, Mengyao Durand, Thibaut Mori, Greg Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning This paper presents FairNVT, a lightweight debiasing framework for pretrained transformer-based encoders that improves both representation and prediction level fairness while preserving task accuracy. Unlike many existing debiasing approaches that address these notions separately, we argue they are inherently connected: suppressing sensitive information at the representation level can facilitate fairer predictions. Our approach learns task-relevant and sensitive embeddings via lightweight adapters, applies calibrated Gaussian noise to the sensitive embedding, and fuses it with the task representation. Together with orthogonality constraints and fairness regularization, these components jointly reduce sensitive-attribute leakage in the learned embeddings and encourage fairer downstream predictions. The framework is compatible with a wide range of pretrained transformer encoders. Across three datasets spanning vision and language, FairNVT reduces sensitive-attribute attacker accuracy, improves demographic-parity and equalized-odds metrics, and maintains high task performance. |
| title | FairNVT: Improving Fairness via Noise Injection in Vision Transformers |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2604.16780 |