SANER: Annotation-free Societal Attribute Neutralizer for Debiasing CLIP
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915295286788096 |
|---|---|
| author | Hirota, Yusuke Chen, Min-Hung Wang, Chien-Yi Nakashima, Yuta Wang, Yu-Chiang Frank Hachiuma, Ryo |
| author_facet | Hirota, Yusuke Chen, Min-Hung Wang, Chien-Yi Nakashima, Yuta Wang, Yu-Chiang Frank Hachiuma, Ryo |
| contents | Large-scale vision-language models, such as CLIP, are known to contain societal bias regarding protected attributes (e.g., gender, age). This paper aims to address the problems of societal bias in CLIP. Although previous studies have proposed to debias societal bias through adversarial learning or test-time projecting, our comprehensive study of these works identifies two critical limitations: 1) loss of attribute information when it is explicitly disclosed in the input and 2) use of the attribute annotations during debiasing process. To mitigate societal bias in CLIP and overcome these limitations simultaneously, we introduce a simple-yet-effective debiasing method called SANER (societal attribute neutralizer) that eliminates attribute information from CLIP text features only of attribute-neutral descriptions. Experimental results show that SANER, which does not require attribute annotations and preserves original information for attribute-specific descriptions, demonstrates superior debiasing ability than the existing methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_10202 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SANER: Annotation-free Societal Attribute Neutralizer for Debiasing CLIP Hirota, Yusuke Chen, Min-Hung Wang, Chien-Yi Nakashima, Yuta Wang, Yu-Chiang Frank Hachiuma, Ryo Computer Vision and Pattern Recognition Large-scale vision-language models, such as CLIP, are known to contain societal bias regarding protected attributes (e.g., gender, age). This paper aims to address the problems of societal bias in CLIP. Although previous studies have proposed to debias societal bias through adversarial learning or test-time projecting, our comprehensive study of these works identifies two critical limitations: 1) loss of attribute information when it is explicitly disclosed in the input and 2) use of the attribute annotations during debiasing process. To mitigate societal bias in CLIP and overcome these limitations simultaneously, we introduce a simple-yet-effective debiasing method called SANER (societal attribute neutralizer) that eliminates attribute information from CLIP text features only of attribute-neutral descriptions. Experimental results show that SANER, which does not require attribute annotations and preserves original information for attribute-specific descriptions, demonstrates superior debiasing ability than the existing methods. |
| title | SANER: Annotation-free Societal Attribute Neutralizer for Debiasing CLIP |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.10202 |