FairRAG: Fair Human Generation via Fair Retrieval Augmentation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914742242639872 |
|---|---|
| author | Shrestha, Robik Zou, Yang Chen, Qiuyu Li, Zhiheng Xie, Yusheng Deng, Siqi |
| author_facet | Shrestha, Robik Zou, Yang Chen, Qiuyu Li, Zhiheng Xie, Yusheng Deng, Siqi |
| contents | Existing text-to-image generative models reflect or even amplify societal biases ingrained in their training data. This is especially concerning for human image generation where models are biased against certain demographic groups. Existing attempts to rectify this issue are hindered by the inherent limitations of the pre-trained models and fail to substantially improve demographic diversity. In this work, we introduce Fair Retrieval Augmented Generation (FairRAG), a novel framework that conditions pre-trained generative models on reference images retrieved from an external image database to improve fairness in human generation. FairRAG enables conditioning through a lightweight linear module that projects reference images into the textual space. To enhance fairness, FairRAG applies simple-yet-effective debiasing strategies, providing images from diverse demographic groups during the generative process. Extensive experiments demonstrate that FairRAG outperforms existing methods in terms of demographic diversity, image-text alignment, and image fidelity while incurring minimal computational overhead during inference. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_19964 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FairRAG: Fair Human Generation via Fair Retrieval Augmentation Shrestha, Robik Zou, Yang Chen, Qiuyu Li, Zhiheng Xie, Yusheng Deng, Siqi Computer Vision and Pattern Recognition Computers and Society Machine Learning Existing text-to-image generative models reflect or even amplify societal biases ingrained in their training data. This is especially concerning for human image generation where models are biased against certain demographic groups. Existing attempts to rectify this issue are hindered by the inherent limitations of the pre-trained models and fail to substantially improve demographic diversity. In this work, we introduce Fair Retrieval Augmented Generation (FairRAG), a novel framework that conditions pre-trained generative models on reference images retrieved from an external image database to improve fairness in human generation. FairRAG enables conditioning through a lightweight linear module that projects reference images into the textual space. To enhance fairness, FairRAG applies simple-yet-effective debiasing strategies, providing images from diverse demographic groups during the generative process. Extensive experiments demonstrate that FairRAG outperforms existing methods in terms of demographic diversity, image-text alignment, and image fidelity while incurring minimal computational overhead during inference. |
| title | FairRAG: Fair Human Generation via Fair Retrieval Augmentation |
| topic | Computer Vision and Pattern Recognition Computers and Society Machine Learning |
| url | https://arxiv.org/abs/2403.19964 |